Tag : load-testing
There are a number of different common types of load tests. They each serve different purposes, and vet different capabilities.
A Real-World Concurrency test simulates normal production-like usage and determines the application’s scalability. This test is probably the single most important test for vetting a site’s capabilities.
Purpose: Vets the application for a certain number of users with production-like traffic and verifies performance characteristics.
Detects: Bottlenecks, issues including inadequate web server connections, CPU, memory, keepalives, inefficient queries and algorithms, load-balancer issues, worker thread/process issues, database connections, query problems and more.
A Stress Test verifies the websites capabilities when pushed passed expected capacity.
Purpose: Determine upper limit of system capacity and the next expected bottleneck.
Detects: Website breaking point, website degradation profile under increasing load.
A Failover Stress Test verfies website performance when resources are taken away and recovered.
Purpose: Determine how failover conditions are handled including reduced web server, app server and database offlining, as well as recovery from changes in these conditions.
Detects: Correct behavior in the face of reduced capabilities, load balancing, automated recovery, error handling.
An Endurance Test, also known as a soak test, involves testing a system with typical production load for a longer period of use.
Purpose: Verify the application can handle extended usage without resource scarcity issues.
Detects: Resource utilization problems over time where usage of a finite resources is exhausted over time. Typical problems include memory leaks, file handle leaks, database connection recycling, unrolled log files absorbing all disk space.
A Bandwidth Test verifies the amount of bandwidth that your web server tier can pass. This test uses a large image hosted on your web server (not a CDN!) to determine the throughput your website can push through. A Bandwidth Test verifies the promised bandwidth can be delivered given your current configuration. By running an isolated bandwidth test, other tests may be safely run without downloading assets like images so long as the implied bandwidth requirements of those tests is within your established bandwidth-ceiling.
Purpose: Verifies your web server and networking capabilities in isolation.
Detects: Limitations with web hosting providers, network misconfiguration, load balancers, web server, and connection problems.
A Baseline Test verifies website performance under a minimal-traffic scenario.
Purpose: This test determines best-case performance measurements for page load times.
Detects: Best-case page load time. Issues in testing scripts themselves.
A home page test is a simple test that can verify some basic scalability issues.
Purpose: For static websites, this single-page test provides a decent estimate of a website’s capabilities. For true web applications, a home page test provides fast feedback on some problems.
Detects: Issues on a website’s homepage, as well as common web server issues and some configuration issues.
Scalability-wise, authentication is one of the most critical functions in a web application. Why? For most non-trivial web applications, the majority of their functionality requires prior authentication. The implication here is that the scalability of the system’s authentication provides a hard-ceiling on your website’s scalability. If your users can’t login, they can’t get to your website’s other functionality.
What is an authentication script in the load/performance testing world? Very simply, it is a targeted script that logs in and logs out of your web application. The authentication script is used to simulate a portion of the load the application experiences during normal usage.
Testing a website’s authentication performance correctly can be challenging. What are the key considerations in an authentication script?
To isolate authentication functionality, a login script should touch as few other pages as possible. If any extraneous pages are included, they should be chosen to be the least impactful from a resource perspective. It is common for web applications to redirect to the last page a user was on before they logged in. In this case, the page should be a fairly simple page, with as little on-page business logic or database calls as possible.
On the same token, scripts other than the login script should touch the login functionality as seldomly as possible, and should not be logging in and out with each iteration. Scripts that login and out on each iteration overweight and overstress the authentication components.
Isolating the authentication functionality achieves several things
For many sites, the login script (in pseudocode) is simply
post credentials to /login,
verify landing page is in authenticated state
verify page is in unauthenticated state
An authentication script should verify that the landing page post-login is in an authenticated state–as opposed to displaying an error, or remaining unauthenticated. The easiest way to accomplish this for most sites is to check the rendered HTML for the username if it is displayed. This isn’t always possible, but provides a quick and easy check. Alternately, check for content that is only present when a user is authenticated.
Additionally, after logging out, the post-logout page should be checked to confirm the username is absent.
Databanking and Caching
At the application-level, applications often cache recent users. Because of this, to make an authentication script’s load characteristics realistic, it needs to authenticate with many different users. So when creating a login/logout script, it must be databanked (aka data-driven) with enough records to simulate normal usage and prevent the application from surviving off cached records, which would skew your results with an overly positive performance result.