Final Paper
How I Performance Test Digital website in a single day from end user perspective
Abstract
Customers have become impatient & curious – Searching answers to their queries on internet through various means including various websites. Slower performance means losing a potential customer
As we move on to digital space, websites become heavy in nature. Heavier the website, difficult it is to maintain the performance of various web-pages. Most of the commercial tools can measure http request / response but fail heavily when it comes to measuring browser rendering times. Having HTML 5 objects containing videos, audios, etc makes it further more difficult to script for performance testing. In this scenario if there is an acquisition where we have to have quick turn around for various content rich websites' performance assessment, it becomes a nightmare for the performance team. To add to it, if the website undergo changes every hour, performance evaluation becomes extremely challenging. Most of the performance testing tool consider http request response but not browser rendering time.
Maintainability of performance test script is always a challenge if the environment is quite volatile, especially when it uses HTML 5 objects and browser rendering end-events.
To bridge the gap, a tool was innovated (by Rajat Taheem) Capable of rendering actual page load times of content rich web-pages while performance testing it.
Scripting Challenges
Scripting is typically complex for content rich websites containing video, flash, etc. Each performance testing tool has its own complexities & challenges for creating & maintaining scripts.
Cookies play an important role in any web-page. They are designed to hold a modest amount of data specific to a particular client and website, and can be accessed either by the web server or the client computer. This allows the server to deliver a page tailored to a particular user, or the page itself can contain some script which is aware of the data in the cookie and so is able to carry information from one visit to the website (or related site) to the next. Cookies need to be handled except for GUI based protocols in each & every tool & comprise of around 35% of scripting time.
Capturing near-user experience is always a challenge. Client side scripting needs special attention. Whether it’s a new site or mobile app you’re launching in today’s competitive online market, the user experience test is an invaluable tool for mitigating risk and building better products.
The complexity of modern applications makes it difficult to implement a realistic and accurate user experience test. Capturing response time of pages containing video streaming objects is difficult.
Commercial tools do have variety of options to capture response time of various web-pages from end-event perspective. However, this requires amalgamation of various tools & licenses to measure response time correctly. This can be a very costly affair if we need to carry various tests.
Commercial Performance testing tools have their own browser while simulating load hence settings of browsers can pose potential challenges while load generation as the client side scripting can confuse the server for right set of response.
Special skill sets are required for scripting & executing in each different tool. Considering the learning curve required to work with different tools & web-pages, there can be very low possibility to find out response time anomalies. On top of it, Script maintenance required for these websites if it undergo Agile development methodology can pose another level of challenge leading to doubtful response times.
Load generation from cloud machines need additional tool installations time & licenses increasing the overall cost of the performance assessment.
When it comes to crawling the website, this can done either manually or using additional tool. Gathering all URLs of a website to test is a task in itself – Hence scripting for a website can be next to impossible
What is LTI Crawloader
LTI Crawloader is a light-weight cloud-ready performance testing utility developed at LTI Performance Labs. It is capable of performance testing content rich websites (Including videos, flash, etc).
Developed using Windows Powershell scripting this tool is capable to crawl a website & generate load on various web-pages based on testing inputs.
Highlight of this tool is that it is Script-less performance testing tool. It doesn’t require any scripting or configuration or installation for it to run. No special licensing or special installation needed – even on cloud. It’s a cloud ready solution.
Capable of running on any windows machine without any special requirement makes it the most wanted tool by our team & by our customers. It just has a bat file to trigger the performance test which in turn triggers the powershell script file.
Total size of the files are less than 150 KB making it portable choice in performance testing group. Since it uses machine’s IE Browser to generate load, it is not dependent on any API for any of its activities. And since there are very low dependencies, chances of error go down drastically.
Though powershell is an interpreted language, care has to been taken to induce multiple level of error handling along with resource handling of the load-generator to ensure we get best possible response time along with graceful exists during errors.
Technically this reduces the overall time of 3 weeks (minimum period) to whoopee 1 day, making it an ideal choice for websites that undergo regular updates each day.
How does LTI Crawloader work?
LTI Crawloader creates IE handle for various activities during the entire execution. It uses IE to open the home URLs and then extracts the various URLs of the websites. It uses same IE to randomly generate load using these extracted URL. Random order ensures near-real user experience as we are never sure what a user is going to click next.
Consolidated report is prepared for each URL with avg response time & Exceptions. Exceptions occur each time there is an issue either with tool or with the URL / page. This helps performance testers to further analyse the matter & take corrective actions as need be.
It uses IE of the system and its settings including proxy & caching. This ensures no extra configuration has to be done & response times are not altered due to wrong configurations. It bundles optimum monitoring of the load-generator machine & auto-corrective measures in case of overloading of load-generator for best results.
How LTI Crawloader help?
It simulates IE objects over http requests for near user-experience and hence real end user experience is captured including what a novice user would see on his/her screen.
It works flawlessly on Streaming videos, images, flash, etc on a web-page and hence is a best in breed option.
Since it has its own mechanism to watch for Optimum CPU / RAM utilization, it makes itself a best option for performance expert.
No tool or intermediate / API licensing required to operate this tool & no prior training is required for the new onboarded expert to work with this tool, making it the best choice for every team.
It has the capacity to simulate as many users as much your system configuration can handle. Since each browser can be heavy and consume lot of CPU & RAM, since it can have media content, it becomes difficult to manage large load from small machines. We need to beef up the machine’s CPU & RAM to ensure smooth performance test executions.
Following diagram shows how LTI Crawloader is different than normally available performance testing tool.
Screenshots:-
Following are the screenshots of the utility:-
a Initial input box
b Utility in action:-
c Output:-
About the Author:-
Ø Over 12 years of Industrial IT experience, currently playing various roles including Practice Manager, Test Solution Architect, Project Manager, Project Lead, Onshore co-ordinator, Recruitment co-ordinator, Trainer, etc (at times working on various accounts/projects simultaneously)
o Performance Testing (e.g. Load, Stress, Endurance, End User Experience, etc) using majorly commercial tools
o Application Life Cycle Management (Infrastructure/Application Performance Monitoring) for live applications & during performance testing with alert mechanism for various breaches
o Front end engineering of web applications
o Automated Functional testing (Test Automation)
Ø Rich Experience in all activities pertaining to Performance Testing including ‘Requirement Gathering’, ‘Solution-ing’, liaising with various teams for various purposes, test planning, test execution, report preparation, performance issue life cycle, approach preparation, estimations, team setup, lab setup, etc
Ø Major contribution for creating ‘Organizational Assets’ including ‘Intellectual Properties’, ‘Approaches’, ‘Capability presentations’, ‘Training needs’, ‘Templates’
Ø Active participation in various Conferences (National/International)
Ø Major Contribution to various ‘WhitePapers’ (Published on company websites, conference websites, conference displays, client meets, etc
LinkedIN profile :: http://www.linkedin.com/in/rajattaheem
THANK YOU!