The purpose of testing, and its two types
Testing exists to find errors before users do, and to confirm that the program does what it was meant to do — for every likely input, not just the convenient ones.
The specification distinguishes two types, and the difference is between testing modules of a program during development and testing the program at the end of production.
Iterative testing happens throughout development. Each module is tested as it is written, and if a fault is found the code is corrected and tested again — a repeating cycle of write, test, refine. Because each part is tested in isolation, a fault is easy to locate.
Final, or terminal, testing happens once the program is complete. The whole program is tested against the original requirements, with all the modules working together, to confirm it meets what was asked for. It catches problems that only appear when parts interact — which iterative testing of individual modules cannot reveal.
Refining algorithms is the outcome of testing rather than a separate activity. Once a test reveals a fault or an inefficiency, the algorithm is changed and retested — that loop is what improves a program from working to working well.
- Iterative testing
- testing modules of a program during development, fixing problems as the program is built
- Final testing
- testing the complete program at the end of development, against what it was meant to do
Syntax and logic errors
Logic errors are the harder of the two, precisely because nothing complains. The translator is satisfied and the program runs to completion — it simply produces an answer that is wrong. That is why trace tables from 2.1.2 matter here: following the algorithm by hand shows where the values first go astray.
Selecting suitable test data
The specification defines four kinds of test data, and the definitions are precise. Two of them are nearly identical in effect but differ in data type, and confusing those two is the single most common error in this topic.
Worked example
A program accepts an age between 13 and 19. Produce a test plan with one example of each kind of test data.
- Normal data sits comfortably inside the range and should be accepted — 16 will do.
- Boundary data is of the correct type and sits on the very edge of validity — 13 and 19 are the edges.
- Invalid data is a number, so the right type, but outside the range — 12 or 25.
- Erroneous data is the wrong type altogether — the word "sixteen".
- For each, state the expected result, since a test without an expected result proves nothing.
Answer: Normal: 16 — expected to be accepted. Boundary: 13 — expected to be accepted, being on the edge of the valid range. Invalid: 25 — a number, but out of range, so expected to be rejected. Erroneous: "sixteen" — the wrong data type, so expected to be rejected.
A test plan sets out, for every test: the test data, the reason for choosing it, the expected result, and the actual result when the test is run. If the actual result differs from the expected one, the fault is recorded, corrected and the test repeated.
Boundary data deserves attention because that is where errors cluster. A condition written as if mark < 100 rather than if mark <= 100 works for every normal value and fails only at exactly 100 — which is precisely why the boundary is tested.
Common mistake
Invalid and erroneous are not interchangeable. Invalid data is the correct data type but should be rejected; erroneous data is the incorrect data type. The specification defines them separately, and a question asking for one will not accept the other.
- Normal test data
- typical data that the program should accept and process correctly
- Boundary test data
- data at the very edge of what is acceptable, which is where off-by-one errors show up
- Invalid test data
- data of the right type but outside the acceptable range, which the program should reject
- Erroneous test data
- data of the wrong type entirely, which the program should reject
Before you practise
- Testing finds errors before users do and confirms the program meets its requirements.
- Iterative testing tests modules during development, repeatedly, as they are written.
- Final or terminal testing tests the finished program as a whole against the original requirements.
- A syntax error breaks the grammatical rules and stops the program running or being translated.
- A logic error runs but produces unexpected output.
- Normal data should be accepted without errors.
- Boundary data is of the correct type and on the very edge of being valid.
- Invalid data is the correct data type but should be rejected.
- Erroneous data is the incorrect data type and should be rejected.
- A test plan records the data, the reason, the expected result and the actual result.