Introduction to Data Validation and Verification
Welcome to one of the most important chapters in AS 2: Fundamentals of Digital Technology! Whenever humans or systems enter data into a computer, mistakes can happen. You might accidentally mistype your age as 200 instead of 20, or spell your name incorrectly. In the computing world, there is a famous rule: Garbage In, Garbage Out (GIGO). This simply means that if poor-quality or incorrect data goes into a system, the output produced by the computer will also be incorrect or useless.
To prevent "garbage" from entering systems, digital technologies rely on two distinct defensive processes: Data Validation and Data Verification. Don't worry if you mix these two terms up at first—they sound very similar, but by the end of these notes, you will know exactly what each one does and how to tell them apart with ease!
Did you know? A simple data entry error once caused a spacecraft to veer off course! Ensuring data is both sensible and accurate saves businesses millions of pounds and keeps critical systems running safely.
Part 1: Data Validation
Data Validation is an automatic check carried out by a computer to ensure that data entered is sensible, reasonable, complete, and within acceptable boundaries.
Crucial Rule to Remember: Validation cannot check if data is completely accurate or true. It only checks that the data follows the preset rules of the system. For example, if your name is John and you type Mary, the computer will accept it because Mary contains only letters and is sensible data, even though it is factually incorrect for you!
Common Validation Checks
Below are the standard validation checks you need to master for your CCEA AS exam:
1. Range Check
• What it does: Checks that numerical or date data falls between an upper and a lower limit.
• Everyday Example: A student's exam mark must be between \(0\) and \(100\) (inclusive). An entry of \(105\) or \(-5\) will be rejected.
• Another Example: A month value must be an integer between \(1\) and \(12\).
2. Length Check
• What it does: Checks that data has an exact number of characters, or falls within a minimum and maximum character count.
• Everyday Example: A UK mobile phone number must be exactly \(11\) digits long. A password must contain at least \(8\) characters.
3. Presence Check
• What it does: Checks that a required field has not been left completely empty or blank.
• Everyday Example: When registering for an online account, the Email Address field is mandatory. If you try to submit the form without entering an email, an error alert stops you.
4. Format Check (Picture Check)
• What it does: Checks that data follows a specific, predefined pattern of letters, numbers, and symbols.
• Everyday Example: A date format must follow \(DD/MM/YYYY\) (e.g., \(25/12/2024\)).
• Another Example: A National Insurance number follows the pattern of two letters, six numbers, and one letter (e.g., \(QQ123456A\)).
5. Type / Character Check
• What it does: Checks that the data entered contains only the permitted character types (e.g., letters only, numbers only, or no special symbols).
• Everyday Example: A Surname field should only contain alphabetic letters (no numbers). A Quantity Ordered field should only contain numerical digits.
6. Lookup Check
• What it does: Compares the entered value against an existing list of valid, predefined values (often implemented as a drop-down menu).
• Everyday Example: Selecting a UK county from a drop-down list, or choosing a day of the week (Monday to Sunday).
7. Consistency Check
• What it does: Compares two or more related fields to make sure they do not contradict each other.
• Everyday Example: If a user selects Title: Mr, but enters Gender: Female, the consistency check flags a mismatch.
• Another Example: If a delivery date is entered as earlier than the order placement date.
8. Check Digit
• What it does: An extra digit added to the end of a long number (such as an ISBN on a book or a barcode). The computer performs a mathematical calculation on all previous digits and compares the result to the check digit. If they match, the number was scanned/entered correctly.
• Common Method: Modulo-11
Let's see how a check digit calculation works step-by-step:
1. Each digit in the code is multiplied by a predetermined "weight".
2. All the resulting products are added together to find a total sum.
3. The sum is divided by \(11\) to find the remainder: \(\text{Remainder} = \text{Sum} \pmod{11}\).
4. The check digit is calculated as \(11 - \text{Remainder}\). If the result is \(10\), an 'X' is often used.
• Why is it useful? It instantly detects human errors such as typing the wrong number or swapping two adjacent digits (transposition errors).
Key Takeaway for Validation
Validation = Sensible? It is an automatic computer process that checks whether data follows logical rules, boundaries, and formats.
Part 2: Data Verification
Data Verification is the process of checking that the data entered into the computer exactly matches the original source document, or that data has not been corrupted during data transmission.
While validation asks "Is this data sensible?", verification asks "Is this data an exact copy of the source?"
Methods of Verification During Data Entry
When humans type data from paper documents into a computer, two main verification methods are used:
1. Double Entry (Double Keying)
• How it works: The same data is typed into the system twice (either by two different operators or by the same operator at different times). The computer system then compares the two entries character by character. If there is any discrepancy, the user is prompted to re-enter or correct the field.
• Everyday Example: When you set a new password, you are asked to type it twice to make sure you didn't make a typing error the first time.
2. Visual Check (Proofreading)
• How it works: The user manually looks at the data displayed on the screen and compares it word-for-word against the original paper document or source sheet.
• Everyday Example: Before confirming an online plane ticket purchase, a confirmation screen asks you to carefully read your full name, passport number, and travel dates to ensure they match your physical passport.
Methods of Verification During Data Transmission
Data can become corrupted when travelling across a network due to electrical interference or signal noise. Systems use verification methods to catch these transmission errors:
1. Parity Checks (Odd or Even Parity)
• How it works: A byte contains \(8\) bits: \(7\) data bits and \(1\) parity bit (usually at the start or end).
• Even Parity: The parity bit is set to \(0\) or \(1\) so that the total number of \(1\text{s}\) in the byte is an even number.
• Odd Parity: The parity bit is set so that the total number of \(1\text{s}\) in the byte is an odd number.
• Example (Even Parity):
Suppose the \(7\) data bits are: \(1 \ 0 \ 1 \ 1 \ 0 \ 0 \ 0\).
Count the \(1\text{s}\): There are \(3\) ones (an odd number).
To make the total count even, the parity bit is set to \(1\).
The transmitted \(8\)-bit byte becomes: \(1 \ 1 \ 0 \ 1 \ 1 \ 0 \ 0 \ 0\) (which has \(4\) ones = even).
If a bit flips during transmission (e.g., arriving with \(3\) ones), the receiver knows an error occurred!
2. Checksums (Hash Totals)
• How it works: Before transmission, an algorithm calculates a numerical value (a checksum) based on the contents of the entire block of data. This checksum is sent alongside the data. Upon arrival, the receiving computer recalculates the checksum using the exact same algorithm. If the calculated checksum matches the received checksum, the data arrived intact.
Key Takeaway for Verification
Verification = Accurate Copy? It checks that entered or transmitted data is an exact duplicate of the source with no copying errors or transmission corruptions.
Part 3: Comparing Validation and Verification
Students often lose marks in exams by confusing these two concepts. Here is a clear comparison to keep them straight in your head:
Memory Tip to Remember the Difference:
• Validation = Value is Acceptable and Logical.
• Verification = Verifying Exact Representation of the original.
Comparison at a Glance
Data Validation:
• Performed by: The computer program/software automatically.
• Main Purpose: Checks if data is sensible, reasonable, and follows rules.
• Can it catch wrong data that looks valid? No (e.g., entering the wrong valid surname will pass).
• Key Examples: Range check, Length check, Format check, Presence check, Check digit.
Data Verification:
• Performed by: Humans (visual check), software (double entry), or network protocols (parity/checksum).
• Main Purpose: Checks if data accurately matches the original source data.
• Can it catch wrong data that looks valid? Yes, because it compares the entry directly against the source document.
• Key Examples: Double entry, Proofreading, Parity check, Checksum.
Common Mistakes to Avoid in the Exam
• Mistake 1: Saying "Validation checks if data is correct."
Correction: Never use the word "correct" for validation! Always write "Validation checks if data is sensible, reasonable, or conforms to set rules."
• Mistake 2: Thinking a Check Digit is verification.
Correction: A Check Digit is calculated by a computer using a mathematical rule, making it a validation check, even though it helps detect typing mistakes.
• Mistake 3: Confusing Double Entry with Presence Check.
Correction: A presence check simply ensures a box isn't empty; double entry forces you to type the value twice to compare the two inputs for equality.
Part 4: Quick Summary & Revision Checklist
Before you move on to past paper questions, check that you can:
• Define Data Validation and state its primary purpose.
• Explain at least six validation checks (Range, Length, Presence, Format, Type, Lookup, Check Digit, Consistency) with clear examples.
• Explain how a Check Digit (such as Modulo-11) works.
• Define Data Verification and explain how Double Entry and Visual Checks work.
• Explain how Parity Bits and Checksums verify data during transmission.
• Clearly explain the difference between data being valid and data being accurate.