← VCE Data Analytics U4O1

Checking end-to-end data integrity from collection to visualisation

Following data through every stage, from collection to chart, and confirming at each stage that it is unchanged and correct, for example by matching record counts and totals.

Part of: VCE Data Analytics - U4O1 - Development and Evaluation — Module 83, ACARA Digital Technologies Years 9–10 — Acquiring and Validating Data — Module 111, QCE Digital Solutions Unit 4 — Complex Digital Data Exchange Problems and Solution Requirements — Module 85

Understanding Checking end-to-end data integrity from collection to visualisation in VCE Data Analytics U4O1

What it does: Looking up one record in the source lets the student compare it with the bar or point on the chart. If they match, that value is unchanged all the way to the visualisation. A written log of each check gives evidence that integrity was confirmed at every stage, and it lets others repeat the checks. End-to-end means every stage, from collection to visualisation, not just the last one. Matching counts shows nothing went missing or was duplicated. It does not prove each value is correct, so you still check totals and values. A total that changes between stages signals a problem, such as missing rows, wrong filters or a formula error. Find it and fix it before you build the chart.

Common mistakes: The line uses == (comparison) instead of = (assignment), so records_in is never stored. It should be records_in = 120. The record count of raw is compared with itself, so the check is always True. It should compare len(raw) with len(clean).

Class: _____________________  ·  Name: _____________________  ·  Date: _________

Free printable worksheet with full answer key — or add this module to a class and your students practise it interactively.

🖨 Print worksheet + answer key Add Checking end-to-end data integrity from collection to visualisation to a class →

This module's key terms are also in a printable crossword (Modules 63–83) and word search (Modules 63–105).

Take this worksheet further — free with Cloodit

Each concept is reinforced from every angle — learned, then applied, then tested.

Learning
Duo
What does end-to-end integrity checking involve?
Following the data from collection to the final chart and confirming it is unchanged and correct at each stageChecking the data only once, after the final chart has been made
Matches
Only check the total on the final chart and skip the earlier stages
↓
End-to-end checking must happen at every stage. Checking only the end cannot show where an error was introduced.
Applying
Meanings
Term
End-to-end integrity checking
Meaning
Following data through every stage, from collection to chart, and confirming at each stage that it is unchanged and correct, for example by matching record counts and totals.
Practice
=IF(COUNT(B2:B9)=COUNTA(A2:A9),"Counts match","Check data")
Testing
Gaps
=(Raw!A:A)=COUNTA(Clean!A:A)
Check Answer
Spots
1records_in == 120
2records_out = 120

Cloodit turns curriculum-aligned coding content into journeys students work through one biome at a time.

The 'Dumbo Octopus' badge — Module Maker, one of Cloodit's 106 collectible badges
Students earn collectable badges

Over 100 of them, across seven natural biomes.

Study Buddy
Students build a real study resource

Turns their work into revision material ready for Flash Cards and Exam study.

Diamond Dash — one of Cloodit's live, competitive Sprint games
Students lock in with Sprints

Live, competitive practice for the start of the lesson or a teacher-led lesson finisher.

A Cloodit certificate of achievement, ready to personalise and print
Custom printable certificates

Honour your top performers, or encourage a student who's trying hard — ready to print in seconds.

Add to class →

Quick Questions

1. What does end-to-end integrity checking involve?
A. Following the data from collection to the final chart and confirming it is unchanged and correct at each stage ✓
B. Checking the data only once, after the final chart has been made
End-to-end means every stage, from collection to visualisation, not just the last one.
2. You collected 250 survey responses. After cleaning, the spreadsheet holds 250 records. What does this record count check show?
A. No records were lost or added between collection and cleaning ✓
B. The data is guaranteed to be 100% accurate
Matching counts shows nothing went missing or was duplicated. It does not prove each value is correct, so you still check totals and values.
3. Raw data has a sales total of $12,400, but the pivot table total is $11,900. What should you do?
A. Ignore it, because small differences do not matter in a chart
B. Investigate the stage where the $500 difference appeared and fix it ✓
A total that changes between stages signals a problem, such as missing rows, wrong filters or a formula error. Find it and fix it before you build the chart.
4. Which check is an example of tracing a value to the chart?
A. Finding the Term 2 bar value in the chart and matching it to the source cell it came from ✓
B. Changing the chart colours so the bars look clearer
Tracing means following a specific value back through each stage to its source to confirm it matches.
5. Why should you document your integrity checks?
A. It makes the spreadsheet calculate faster
B. It gives evidence of what was checked, the result and any fixes made at each stage ✓
Documentation records the checks and their outcomes, so others can see the data was verified at every stage.

Match the Pairs

=COUNTA(A2:A9)=COUNTA(Clean!A2:A9) → Compares the number of records in the source with the number in the cleaned data to check none were lost or added
=SUM(B2:B9)-SUM(Summary!B2:B9) → Subtracts the summary total from the source total; a result of 0 shows the total is unchanged between stages
Tracing a value to the chart: pick one value in the source data and find the same value in the chart's data table → =XLOOKUP("Mia",A2:A9,B2:B9)
Documenting the checks: recording each stage, what was checked, the expected result and the actual result → Stage: Cleaned | Check: count | Expected: 8 | Actual: 8
Only check the total on the final chart and skip the earlier stages → End-to-end checking must happen at every stage. Checking only the end cannot show where an error was introduced.
Source has 120 records, cleaned data has 115 records, so mark it as passed → The record counts do not match, so 5 records are missing and the check has failed. It should not be marked as passed.

Key Term

End-to-end integrity checking
Following data through every stage, from collection to chart, and confirming at each stage that it is unchanged and correct, for example by matching record counts and totals.

Fill in the Blanks

1. =COUNTA(Raw!A:A)=COUNTA(Clean!A:A)
Hint: Type the function that counts non-blank cells, so the record counts of two stages can be compared.
2. =IF(SUM(Data!C:C)=SUM(Viz!B:B),"OK","Fix")
Hint: Type the function that adds up a column, so the totals at two stages can be checked.
3. =XLOOKUP(A2,Data!A:A,Data!C:C)
Hint: Type the lookup function that finds a matching item so a value can be traced back to the source data.
4. log.append("Totals match: 5210 = 5210")
Hint: Type the list method that adds a new entry to the end of the log so each check is documented.
5. if len(raw) != len(chart_data): print("Records lost")
Hint: Type the comparison operator that detects when the two record counts are different.

Spot the Error

Question 1
A records_in == 120
B records_out = 120
C if records_in == records_out:
D print('Counts match')
Line A — The line uses == (comparison) instead of = (assignment), so records_in is never stored. It should be records_in = 120.
Question 2
A raw = [12, 15, 9, 20]
B clean = [12, 15, 9, 20]
C print(len(raw) == len(raw))
Line C — The record count of raw is compared with itself, so the check is always True. It should compare len(raw) with len(clean).
Question 3
A raw = [12, 15, 9, 20]
B chart = [12, 15, 9, 20]
C print(sum(raw) == len(chart))
Line C — This compares a total with a record count, which are different measures. It should compare sum(raw) with sum(chart).
Question 4
A raw = [40, 25, 35]
B total_raw = len(raw)
C chart_total = 100
D print(total_raw == chart_total)
Line B — len() gives the number of records (3), not the total of the values. It should be sum(raw) so the total can be checked against the chart.
Question 5
A data = {'Mon': 40, 'Tue': 55}
B chart = 55
C print(data['Mon'] == chart)
Line C — The chart bar shows the Tuesday value (55), so the source value traced must be data['Tue'], not data['Mon'].

Student with a class code? Join your class →