Table of Contents
1. A Fault Model to Guide Software Testing.
The Purpose of Software Testing. Understanding Software Behavior.
Understanding Software's Environment. The Human User. File System
User. The Operating System User. The Software User. Understanding
Software's Capabilities. Testing Input. Testing Output. Testing
Data. Testing Competition. Summary and Conclusion. Exercises.
II. USER INTERFACE ATTACKS.
2. Testing from the User Interface: Inputs and Outputs.
Using the Fault Model to Guide Testing. Exploring the Input Domain.
First Attack: Apply inputs that force all the error messages to
occur. Second Attack: Apply inputs that force the software to
establish default values. Third Attack: Explore allowable character
sets and data types. Fourth Attack: Overflow input buffers. Fifth
Attack: Find inputs that may interact and test various combinations
of their values. Sixth Attack: Repeat the same input or series of
inputs numerous times. Exploring Outputs. Seventh Attack: Force
different outputs to be generated for each input. Eighth Attack:
Force invalid outputs to be generated. Ninth Attack: Force
properties of an output to change. Tenth Attack: Force the screen
to refresh. Conclusion. Exercises. References. 3. Testing from
the User Interface: Data and Computation.
Testing Inside the Box. Exploring Stored Data. Eleventh Attack:
Apply inputs using a variety of initial conditions. Twelfth Attack:
Force a data structure to store too many/too few values. Thirteenth
Attack: Investigate alternate ways to modify internal data
constraints. Exploring Computation and Feature Interaction.
Fifteenth Attack: Force a function to call itself recursively.
Sixteenth Attack: Force computation results to be too large or too
small. Seventeenth Attack: Find features that share data or
interact poorly. Conclusion. Exercises.
III. SYSTEM INTERFACE ATTACKS.
4. Testing from the File System Interface.
Attacking Software from the File System Interface. Media-based
Attacks. First Attack: Inject faults that simulate memory access
problems. Second Attack: Inject faults that simulate network
problems. Third Attack: Damage the media. File-based Attacks.
Fourth Attack: Assign an invalid file name. Fifth Attack: Vary file
access permissions. Sixth Attack: Vary/corrupt file contents.
Exercises. 5. Testing from the Software/OS Interface.
Attacking Software from Software Interfaces. Record-and-Simulate
Attacks. Observe-and-Fail Attacks. Conclusion. Exercises.
6. Some Parting Advice.
You'll Never Know Everything. Bug Hunts. Friday Afternoon Bug
Fests. Conclusion. References.
Annotated Glossary of Programming Terms.
Appendix A. Testing Exception and Error Cases Using Runtime
Introduction. A Mechanism for Runtime Fault Injection. Fault
Selection. Conclusions. Acknowledgments. References. Appendix B.
Using HEAT: The Hostile Environment Application Tester.
Canned HEAT User Guide. The Application Band. The Monitor Band.
Fault Injection Bands and Their Functionality. The Network Band.
Disk Storage. Memory. Appendix C. What is Software Testing? And
Why is it so Hard?
Introduction. The Software Testing Process. Phase One: Modeling the
Software's Environment. Phase Two: Selecting Test Scenarios. Phase
Three: Running and Evaluating Test Scenarios. Phase Four: Measuring
Testing Progress. Conclusion. References. The Software Testing
How to Break Software is a departure from conventional testing in
which testers prepare a written test plan and then use it as a
script when testing the software. The testing techniques in this
book are as flexible as conventional testing is rigid. And
flexibility is needed in software projects in which requirements
can change, bugs can become features and schedule pressures often
force plans to be reassessed. Software testing is not such an exact
science that one can determine what to test in advance and then
execute the plan and be done with it. Instead of a plan,
intelligence, insight, experience and a "nose for where the bugs
are hiding" should guide testers. This book helps testers develop
this insight. The techniques presented in this book not only allow
testers to go off-script, they encourage them to do so. Don't
blindly follow a document that may be out of date and that was
written before the product was even testable. Instead, use your
head! Open your eyes! Think a little, test a little and then think
a little more. This book does teach planning, but in an "on-
the-fly while you are testing" way. It also encourages automation
with many repetitive and complex tasks that require good tools (one
such tool is shipped with this book on the companion CD). However,
tools are never used as a replacement for intelligence. Testers do
the thinking and use tools to collect data and help them explore
applications more efficiently and effectively.
About the Author
James A. Whittaker is a well-known speaker and consultant, as well
as seasoned professor.