Keys to Efficiently and Effectively Running Tests In general, the keys to efficiently and effectively running tests include: • Revisit performance-testing priorities after no more than two days. • Remember to capture and use a performance baseline. • Plan to spend some time fixing application errors, or debugging the test. • Analyze results immediately so that you can modify your test plan accordingly. • Communicate test results frequently and openly across the team. • Record results and significant findings. • Record other data needed to repeat the test later. • At appropriate points during test execution, stress the application to its maximum capacity or user load, as this can provide extremely valuable information. • Remember to validate application tuning or optimizations. • Consider evaluating the effect of application failover and recovery. • Consider measuring the effects of different system configurations.
Additional Considerations Consider the following additional points when running your tests: • Performance testing is frequently conducted on an isolated network segment to prevent disruption of other business operations. If this is not the case for your test project, ensure that you obtain permission to generate loads during certain hours on the available network. • Before running the real test, consider executing a quick “smoke test” to make sure that the test script and remote performance counters are working correctly. • If you choose to execute a smoke test, do not report the results as official or formal parts of your testing. • Reset the system (unless your scenario is to do otherwise) before running a formal test. • If at all possible, execute every test twice. If the results produced are not very similar, execute the test again. Try to determine what factors account for the difference. • No matter how far in advance a test is scheduled, give the team 30-minute and 5minute warnings before launching the test (or starting the day’s testing). Inform the team whenever you are not going to be executing for more than one hour in succession. • Do not process data, write reports, or draw diagrams on your load-generating machine while generating a load because this can corrupt the data. • Do not throw away the first iteration because of script compilation or other reasons. Instead, measure this iteration separately so you will know what the first user after a system-wide reboot can expect. • Test execution is never really finished, but eventually you will reach a point of diminishing returns on a particular test. When you stop obtaining valuable information, change your test. • If neither you nor your development team can figure out the cause of an issue in twice as much time as it took the test to execute, it may be more efficient to eliminate one or more variables/potential causes and try again.