The following activities frequently prove valuable when validating a test environment: • Ensure that the test environment is correctly configured for metrics collection. • Turn off any active virus-scanning on load-generating machines during testing, to minimize the likelihood of unintentionally skewing results data as a side-effect of resource consumption by the antivirus/anti-spyware software. • Consider simulating background activity, when necessary. For example, many servers run batch processing during predetermined time periods, while servicing users’ requests. Not accounting for such activities in those periods may result in overly optimistic performance results. • Run simple usage scenarios to validate the Web server layer first if possible, separately from other layers. Run your scripts without think times. Try to run a scenario that does not include database activity. Inability to utilize 100 percent of the Web server’s processor can indicate a network problem or that the load generator clients have reached their maximum output capacity. • Run simple usage scenarios that are limited to reading data to validate database scenarios. Run your script without think times. Use test data feeds to simulate randomness. For example, query for a set of products. Inability to utilize 100 percent of the Web server’s processor can indicate a network problem or that the loadgenerator clients have reached their maximum output capacity. • Validate the test environment by running more complex usage scenarios with updates and writes to the database, using a mix of test scripts that simulate business actions. • In Web farm environments, check to see if your load tests are implementing Internet Protocol (IP) switching. Not doing so may cause IP affinity, a situation where all of the requests from the load-generation client machine are routed to the same server rather than being balanced across all of the servers in the farm. IP affinity leads to inaccurate load test results because other servers participating in the load balancing will not be utilized. • Work with key performance indicators (KPIs) on all the servers to assess your test environment (processor, network, disk, and memory). Include all servers in the cluster to ensure correct evaluation of your environment. • Consider spending time creating data feeds for the test application. For example, database tables containing production data such as number of users, products, and orders shipped, so that you can create similar conditions to replicate problems in critical usage scenarios. Many scenarios involve running queries against tables containing several thousands of entries, to simulate lock timeouts or deadlocks.
Additional Considerations Consider the following key points when troubleshooting performance-testing environments: • Look for problems in the load-generation clients from which load is simulated. Client machines often produce inaccurate performance-testing results due to insufficient processor or memory resources. Consider adding more client computers to compensate for fast transactions that may cause higher processor utilization; also consider using more memory when this becomes the bottleneck. Memory can be consumed when test data feeds are cached in load generators, or by more complex scripting in load tests.