Retesting everything after a fix

Name:

Date:

Class:

Quick Questions

Shade in the A or B circle for each question.

1. You fix a bug and the test that failed now passes. What should you do next?

A Stop testing because the bug is fixed

B Rerun the failed test and all the earlier tests

2. True or false: A fix to one part of a program can cause a test that used to pass to fail.

A True

B False

3. Which one describes regression testing?

A Rerunning old tests after a change to check nothing that used to pass has broken

B Writing brand new tests for a program you have not run yet

4. Mia fixes a bug in a discount function. Which retest plan is best?

A Rerun the failed test plus every earlier test

B Rerun only the test that failed

5. After a fix, Test 3 now passes but Test 1, which passed yesterday, now fails. What does this show?

A Test 1 must be wrong and can be ignored

B The fix broke code that used to work

Match the Pairs

Draw a line connecting each item on the left to its match on the right.

Left column		Right column
1regression testing		Aprint(add(2, 3) == 5)
2for t in tests: run(t)		BRerunning the failed test and all the earlier tests after a fix, to check nothing that used to pass is now broken
3After fixing a bug, rerun the test that failed to check the fix actually worked		CAssuming earlier tests still pass is wrong. A fix can break other code, so tests 1, 2 and 4 must be rerun and checked.
4A fix can break code that used to pass, so an earlier passing test can now fail		DOnly the failed test was rerun. The earlier tests were skipped, so the fix may have broken something that used to pass.
5Fix bug -> rerun only the failed test -> release the program		ERerun every saved test case after a fix, not just the one that failed
6Test 3 passes after the fix -> assume tests 1, 2 and 4 still pass		Fprint(is_even(4) == True) # passed before the fix, FAIL after

Definitions

regression testing

Fill in the Blanks

Complete each line of code by filling in the missing parts.

tests = [(2, 4), (3, 9), (0, 0)] for n, expected in _____: print(square(n) == expected)

for n, expected in tests: if square(n) __ expected: print("FAIL", n)

___ run_tests(): for n, e in tests: print(square(n) == e)

def square(n): return n * n _________()

def double(n): return n * 2 assert double(4) == _

Spot the Error

Shade in the letter of the line that contains an error.

Question 1

A def double(n): return n * 2

B tests = [(3, 6)]

C for n, e in tests:

D ok = double(n) == e

E print(n, ok)

Question 2

A def add_one(n): return n + 1

B tests = [(1, 2), (9, 10)]

C for n, e in tests[:1]:

D print(add_one(n) == e)

Question 3

A def square(n): return n * n

B for n, e in [(2, 4), (3, 9)]:

C if square(n) = e:

D print('PASS', n)

E else: print('FAIL', n)

Question 4

A def double(n): return n + 3

B tests = [(0, 0), (3, 6)]

C for n, e in tests:

D print(double(n) == e)

Question 5

A def half(n): return n / 2

B tests = [(4, 2), (10, 5)]

C failed = 1

D for n, e in tests:

E if half(n) != e:

F failed += 1

Answer Key

Retesting everything after a fix

Section A: Quick Questions

B — Rerun the failed test and all the earlier tests

A — True

A — Rerunning old tests after a change to check nothing that used to pass has broken

A — Rerun the failed test plus every earlier test

B — The fix broke code that used to work

Section B: Match the Pairs

regression testing → Rerunning the failed test and all the earlier tests after a fix, to check nothing that used to pass is now broken

for t in tests: run(t) → Rerun every saved test case after a fix, not just the one that failed

After fixing a bug, rerun the test that failed to check the fix actually worked → print(add(2, 3) == 5)

A fix can break code that used to pass, so an earlier passing test can now fail → print(is_even(4) == True) # passed before the fix, FAIL after

Fix bug -> rerun only the failed test -> release the program → Only the failed test was rerun. The earlier tests were skipped, so the fix may have broken something that used to pass.

Test 3 passes after the fix -> assume tests 1, 2 and 4 still pass → Assuming earlier tests still pass is wrong. A fix can break other code, so tests 1, 2 and 4 must be rerun and checked.

Section C: Definitions

regression testing — After fixing a bug, rerun the failed test and all earlier tests, because a fix can break code that used to pass.

Section D: Fill in the Blanks

tests = [(2, 4), (3, 9), (0, 0)]
for n, expected in tests:
    print(square(n) == expected)

for n, expected in tests:
    if square(n) != expected:
        print("FAIL", n)

def run_tests():
    for n, e in tests:
        print(square(n) == e)

def square(n):
    return n * n
run_tests()

def double(n):
    return n * 2
assert double(4) == 8

Section E: Spot the Error

B — After a fix you must rerun the failed test AND all the earlier tests. This list only holds the one test that failed, so the earlier test (0, 0) is never rerun. It should be tests = [(0, 0), (3, 6)].

C — The slice [:1] keeps only the first test, so the rest are skipped after the fix. The loop should go through every test: for n, e in tests:

C — A single = is assignment, not comparison, so this is a SyntaxError and no retests run. It should be if square(n) == e:

A — This 'fix' makes the test 3 -> 6 pass but breaks 0 -> 0, which used to pass. That is why you rerun all earlier tests after a fix. The correct code is return n * 2.

C — The failure counter must start at 0 on every retest run. Starting at 1 reports a failure even when every test passes. It should be failed = 0.