Evaluation strategy: data and methods

Name:

Date:

Class:

Quick Questions

Shade in the A or B circle for each question.

1. Which item belongs in an evaluation strategy's data and methods section?

A Data to collect, the method used, and the criterion it relates to

B Only a list of bugs fixed during testing

2. A criterion says: 'The program processes 1000 records in under 2 seconds.' Which data and method links best?

A Opinions gathered from a survey about how nice the colours are

B Processing times recorded by running the program with timing code

3. A criterion says: 'Users find the interface easy to navigate.' Which method suits it?

A Count the number of lines of code in the program

B Observe users completing tasks and give them a survey

4. Which is a method of data collection?

A Survey

B Criterion

5. Why should each piece of data be linked to a criterion?

A So the data helps you judge whether that criterion was met

B So the data can be collected more cheaply

Match the Pairs

Draw a line connecting each item on the left to its match on the right.

Left column		Right column
1Data: average page load time \| Method: timed test runs \| Criterion: efficiency		ASurvey responses will be collected from users to judge how easy the solution is to use.
2Data: 20 user ratings out of 5 \| Method: survey \| Criterion: usability		BAn evaluation strategy entry that names the data to collect, the method used, and the criterion the data will judge.
3Each piece of data in the strategy is linked to the criterion it helps judge.		CThe data is not linked to any evaluation criterion, so we cannot tell what it will help judge.
4The strategy states the method used to collect each type of data, such as observation.		DCriterion: The program must output correct totals \| Data: test results vs expected output
5Method: survey \| Criterion: usability		EThe data to be collected is missing, so we do not know what the survey will measure.
6Data: number of crashes \| Method: observation		FData: time taken to finish a task \| Method: observe users completing the task

Definitions

evaluation strategy

Fill in the Blanks

Complete each line of code by filling in the missing parts.

plan = {"criterion": "Fast loading", "____": "load time in seconds", "method": "timing test"}

plan = {"criterion": "Easy to use", "data": "ratings from users", "______": "survey"}

plan = {"_________": "Accurate results", "data": "actual vs expected output", "method": "test cases"}

___ plan in strategies: print(plan["criterion"]) print(plan["data"])

methods = ["survey", "interview", "test"] if "survey" __ methods: print("Collects user ratings")

Spot the Error

Shade in the letter of the line that contains an error.

Question 1

A criterion = "Loads in 2 sec"

B data = "Load time in seconds"

C method = "Survey of 10 users"

Question 2

A criterion = "Easy to learn"

B data = "Ratings from 1 to 5"

C method = "Survey of 20 users"

D print("Links to:", method)

Question 3

A criterion = "Data is accurate"

B data = "Time to finish a task"

C method = "Test cases"

Question 4

A data = ["Time", "Clicks"

B method = "Observe 5 users"

C criterion = "Quick to finish"

Question 5

A plan = {}

B plan["data"] = "Error count"

C plan["method"] = Observation

D plan["crit"] = "Few errors"

Answer Key

Evaluation strategy: data and methods

Section A: Quick Questions

A — Data to collect, the method used, and the criterion it relates to

B — Processing times recorded by running the program with timing code

B — Observe users completing tasks and give them a survey

A — Survey

A — So the data helps you judge whether that criterion was met

Section B: Match the Pairs

Data: average page load time | Method: timed test runs | Criterion: efficiency → An evaluation strategy entry that names the data to collect, the method used, and the criterion the data will judge.

Data: 20 user ratings out of 5 | Method: survey | Criterion: usability → Survey responses will be collected from users to judge how easy the solution is to use.

Each piece of data in the strategy is linked to the criterion it helps judge. → Criterion: The program must output correct totals | Data: test results vs expected output

The strategy states the method used to collect each type of data, such as observation. → Data: time taken to finish a task | Method: observe users completing the task

Method: survey | Criterion: usability → The data to be collected is missing, so we do not know what the survey will measure.

Data: number of crashes | Method: observation → The data is not linked to any evaluation criterion, so we cannot tell what it will help judge.

Section C: Definitions

evaluation strategy — A plan listing the data to collect, the method used to collect it (such as a survey, test or observation) and which evaluation criterion each piece of data will help you judge.

Section D: Fill in the Blanks

plan = {"criterion": "Fast loading",
        "data": "load time in seconds",
        "method": "timing test"}

plan = {"criterion": "Easy to use",
        "data": "ratings from users",
        "method": "survey"}

plan = {"criterion": "Accurate results",
        "data": "actual vs expected output",
        "method": "test cases"}

for plan in strategies:
    print(plan["criterion"])
    print(plan["data"])

methods = ["survey", "interview", "test"]
if "survey" in methods:
    print("Collects user ratings")

Section E: Spot the Error

C — The criterion is about load speed, so the data is measured load time. A survey only gives opinions, not measured times. The method should be a timed test, such as method = "Timed test, 5 runs".

D — An evaluation strategy must link the data to an evaluation criterion. The line should print criterion, not method, because the method is how the data is collected, not what the data is judged against.

B — The data must help you judge the criterion. Time to finish a task says nothing about accuracy. The data should be something like the number of wrong results found.

A — The list is missing its closing square bracket, which causes a syntax error. It should be data = ["Time", "Clicks"].

C — The method is a piece of text, so it needs quotation marks. Without them Python looks for a variable called Observation and raises a NameError. It should be plan["method"] = "Observation".