Documenting beta test results

Tip: Choose Save as PDF in the print dialog to download. Use A4 paper size.

Documenting beta test results

Name:
Date:
Class:

Quick Questions

Shade in the A or B circle for each question.

1. Which is the best description of beta test results?
A A record of each scenario, what happened, tester comments and any issue found
B A list of the features the developer plans to add next year
2. Which row belongs in a beta test results table?
A Scenario 2: Add item to cart | Developer thinks the code is neat | Issue: none
B Scenario 2: Add item to cart | Button did not respond | Tester: 'Nothing happened' | Issue: Add button fails
3. A tester says, 'I could not find the Save button.' Where should this be recorded?
A Nowhere, because comments are not part of the results
B In the tester comments column of the results table
4. Which tester comment is more useful to record?
A 'The error message appeared, but it did not tell me which field was wrong.'
B 'It was bad.'
5. Scenario: Log in with a valid password. The result: the app crashed. What goes in the issues found column?
A Login crashes with a valid password
B Login works well

Match the Pairs

Draw a line connecting each item on the left to its match on the right.

Left columnRight column
1Scenario 3: Add item to cart | Result: Button did not respond AA tester comment that records the user's own words and opinion about the experience, which can point to a usability issue
2Tester comment: "I couldn't tell if my order was saved." BIssue found: Login fails when the password contains a space
3Beta test results should list each scenario with what happened, the tester's comments and any issue found C| Scenario | What happened | Comment | Issue found |
4An issue found is written clearly in the results so the developer knows what to fix DThe issue is too vague and does not say which scenario or feature failed, so the developer cannot tell what to fix
5Scenario 2: Search for a product | What happened: Fine | Issue found: (blank) EThe result is vague and leaves out the tester's comment, so the record is incomplete and does not show what actually happened
6Issue found: The app is bad FA results table row that records which scenario was run and what actually happened when the tester did it

Definitions

Beta test results

Fill in the Blanks

Complete each line of code by filling in the missing parts.

1.
row = {"scenario": "Login", "issue": "None"} results.______(row)
2.
for r in results: if r["_____"] != "None": print(r["scenario"])
3.
comment = _____("Tester comment: ") print("Saved:", comment)
4.
issues = ["Crash on save", "Typo"] print("Issues:", ___(issues))
5.
test = {"scenario": "Search", "comment": "Slow"} print(test["_______"])

Spot the Error

Shade in the letter of the line that contains an error.

Question 1
A results = []
B r = {"scenario": "Login"}
C results.add(r)
D print(len(results))
Question 2
A f = open("beta.csv", "w")
B f.write("id,result,issue\n")
C f.write("1;Pass;None\n")
D f.close()
Question 3
A names = ["Login", "Search"]
B res = ["Pass", "Fail"]
C for i in range(len(names)):
D print(names[i], res[i+1])
Question 4
A r = {}
B r["scenario"] = "Login"
C r["comment"] = Too slow
D print(r)
Question 5
A outcome = "Fail"
B if outcome = "Fail":
C issue = "Crash on save"
D print("Issue:", issue)

Answer Key

Documenting beta test results
Section A: Quick Questions
Q1
A — A record of each scenario, what happened, tester comments and any issue found
Q2
B — Scenario 2: Add item to cart | Button did not respond | Tester: 'Nothing happened' | Issue: Add button fails
Q3
B — In the tester comments column of the results table
Q4
A — 'The error message appeared, but it did not tell me which field was wrong.'
Q5
A — Login crashes with a valid password
Section B: Match the Pairs
Q1
Scenario 3: Add item to cart | Result: Button did not respond → A results table row that records which scenario was run and what actually happened when the tester did it
Q2
Tester comment: "I couldn't tell if my order was saved." → A tester comment that records the user's own words and opinion about the experience, which can point to a usability issue
Q3
Beta test results should list each scenario with what happened, the tester's comments and any issue found → | Scenario | What happened | Comment | Issue found |
Q4
An issue found is written clearly in the results so the developer knows what to fix → Issue found: Login fails when the password contains a space
Q5
Scenario 2: Search for a product | What happened: Fine | Issue found: (blank) → The result is vague and leaves out the tester's comment, so the record is incomplete and does not show what actually happened
Q6
Issue found: The app is bad → The issue is too vague and does not say which scenario or feature failed, so the developer cannot tell what to fix
Section C: Definitions
Q1
Beta test results — A record of each beta test scenario, what happened, the tester's comments and any issue found, so the developer can decide what to fix.
Section D: Fill in the Blanks
Q1
row = {"scenario": "Login", "issue": "None"} results.append(row)
Q2
for r in results: if r["issue"] != "None": print(r["scenario"])
Q3
comment = input("Tester comment: ") print("Saved:", comment)
Q4
issues = ["Crash on save", "Typo"] print("Issues:", len(issues))
Q5
test = {"scenario": "Search", "comment": "Slow"} print(test["comment"])
Section E: Spot the Error
Q1
C — Lists have no add() method. To store the scenario's result record in the results table, use results.append(r).
Q2
C — The header row uses commas, so the results row must also use commas to be a valid CSV. Line 3 should be f.write("1,Pass,None\n").
Q3
D — res[i+1] pairs each scenario with the next scenario's result, and the last loop causes an IndexError. Use res[i] so each scenario shows its own result.
Q4
C — The tester's comment is text, so it must be in quotes. Without quotes Python reads Too slow as code and raises a syntax error. It should be r["comment"] = "Too slow".
Q5
B — A single = assigns a value and cannot be used in an if condition. To check whether the scenario failed, use the comparison operator ==.