Program Testing Guide
For most programs, it is practically impossible
to prove that the program is correct on all inputs. Instead,
you must try to convince skeptics that it mostly works right, by
testing the program on various inputs and documenting what you
have done. This documentation is called a test plan, and
you must provide one with each program. The test plan
describes how you intend to test your program, that is, which inputs
you plan to test it on and why those are good choices.
The reason for documenting your testing is so that
- it is clear what a program should produce
- testing is repeatable
The results of running a set of tests is a test log, which
shows the results produced by the tests in the test plan.
The log should demonstrate that each test
produced the output predicted by the test plan.
IMPORTANT:
It is your responsibility to make sure that your program runs
on the machine specified for the assignment (e.g., lab computers,
department Unix servers), even though you may develop your program
on your own computer.
We will test your program for correctness
on input sets of our own choosing and on the computers specified
in the assignment.
Developing a Test Plan:
The programs written for this course will require you to test the
execution of your programs on various sets of inputs to demonstrate
program functionality. These input sets, or test cases, will be of
your own choosing, but the following guidelines should be followed:
- Input sets should be ordered logically, preferably in the same
order as the code being tested. For example, if your program has
multiple branches, you should test one branch completely prior to
beginning the testing of another.
- Whenever possible, choose test cases whose expected output can be quickly
computed in your head (without the need for a calculator).
- When there are only a few possible inputs, test them all. Otherwise,
choose inputs that exercise all branches of the code (for example,
one set of inputs that satisfies the condition in an if statement and a second
set of inputs that doesn't satisfy the condition).
- Be thorough. Pretend that you are an adversary trying to
"trick" the program into giving the wrong answer. For
example, input sets should include zero or negative values even if the
"normal" case is a positive number. However, you are not
obligated to test inputs that violate your stated
assumptions.
If your program does not handle certain inputs, the ASSUMPTIONS
section of the program header must indicate precisely which inputs are
not handled in the program.
Format for Program Test Document:
Test results must be submitted separately on paper, even if the
program may be submitted on disk or electronically. The program test
document should be organized as follows:
- Your name (and the names of any people you work with)
- The course number (e.g., CPSC 211) and your section number
(e.g., 201-203)
- The name of the programming assignment (e.g., Program 1)
- Rationale behind chosen test cases (inputs). Note that
you MUST
include some justification for each test case,
even if it is something as simple as
testing the program on odd or even numbers.
- Printouts of output files for each separate input set
(i.e., test logs).