RBT SUCCESS
test coverage is challenging. The complexity of modern applications—in particular, distributed application—makes it more difficult to cover all of the possible scenarios due to the sheer number of alternative paths through the application. Any attempt to systematically (or automatically) consider all possible combinations results in a set of test cases that is too large to manage and takes a very long time to execute, which can mean further delays in certification. In addition, change management has become unwieldy in many organizations, as application requirements change hourly or daily. However, without the ability to properly manage these changes, traceability between requirements and test cases is often lost, making it even harder to ensure continuous test coverage.
RBT Best Practices: The Big Three Requirements-based testing is an approach that is implemented through well-defined processes that target the root causes of quality problems mentioned previously. Even if your organization is not using the practice today, you’re probably familiar with the key processes within RBT, highlighted in Figure 2. In addition to existing testing processes, organizations should adopt the “Big Three” best practices noted above to be successful with RBT: 1. Test early and often. As soon as requirements, design and code artifacts are ready, review them against business objectives, requirements, use cases and test cases. Test early and frequently because defects detected in earlier development phases are cheaper to fix and result in significantly fewer surprises when the code is actually delivered. Make testing a parallel activity to the development process to make all stakeholders aware of quality objectives. By doing this, testing will be seen less as a bottleneck activity— performed after code is delivered under extreme time pressures—and more as a key contributor in the overall quality process. Since software project success can be directly connected to a solid understanding and articulation of requirements, organizations leveraging RBT place a great deal of importance on the quality of requirements. For example, best practices within the RBT requirements quality process include those DECEMBER 2006
described in “Requirements QC.” 2. Test with your head, not your gut. Most organizations value experienced testers and count on you to catch what others might overlook. While being a single “go-to quality expert” may be great for an individual contributor’s career, it can also put unnecessary pressure on that person and the other members of the team because the organization is relying on instinct rather than collective knowledge to achieve quality goals. Today, test case design is rarely performed in a systematic and rigorous manner. Rather, many tests are designed based on gut feel and intuition, leading to unpredictable quality. Best practices within RBT require organizations to support methodical and systematic test-case design to ensure predictable test coverage. A rigorous, logical test-case design process facilitates test case design that doesn’t depend on the skill or experience of specific testers. Rather, it helps organizations inject repeatability and consistent methodology to the test planning process to make test coverage more predictable. Organizations using this method also can apply various optimization techniques to produce the minimum number of test cases required for sufficient test coverage, making the testing cycle faster and more manageable. In most organizations, textual requirements are very important, given that almost all business analysts and domain experts use natural language to express requirements. However, using textual requirements can make it more difficult to achieve and validate high test coverage. As a result, testers relying exclusively on natural language can use only their intuition to claim that the set of test cases that they designed covers 100 percent of the functionality captured in the requirements. With nonstructured textual requirements, these testers lack a formal way to show complete test coverage, so often “corners” or even major scenarios remain untested until defects occur in production. To systematically achieve high test coverage, organizations must create more formal and structured representations of requirements. Once this is done, it’s possible to define “logical” (skeletal) test cases and ensure optimal coverage of requirements, as these
REQUIREMENTS QC Validation of requirements against business objectives. This optimizes project scope by ensuring that each requirement satisfies at least one business objective. If the requirements and business objectives don’t match, refinement is necessary. Review of requirements by stakeholders. Feedback of users and domain experts should be used to refine the requirements before additional work is done. Development of use cases. Map requirements against a task- or interaction-oriented view of the system. If one or more use cases can’t be addressed by the requirements, the requirements are not complete. Application of language analysis techniques. Detect problematic phrases in the requirements and fix them to guarantee consistency and clarity. A problematic phrase—language that is ambiguous, unclear or inconsistent—can be interpreted in multiple ways by different people, which can lead to confusion and errors later on. Ambiguous terms also result in requirements that are not testable.
logical test cases eventually evolve into the actual set of tests that are run against the system. Multiple techniques can be used to provide structure and formality to natural language requirements. They reveal cause-effect relationships embedded within requirements, expressing requirements as a set of conditions (causes) and resulting actions (effects). Cause-effect charting is one of these techniques. Another way to achieve similar goals is to express requirements as flow charts, since they naturally depict precedence dependency between actions as well as conditional branching of activities. Once expressed in the structured manner described above, it’s much easier to derive the equivalent test cases for the requirements. A set of logical test cases can be defined (manually or automatically), which is exactly equivalent to the functionality captured in the requirements. However, this set of test cases may include many redundant cases (overlap with other test cases). To optimize the number of test cases but still provide full coverage, techniques such as decision (truth) tables www.stpmag.com •
29