Welcome to Software Testing!
Hello and welcome! In this chapter, we are going to explore one of the most crucial stages in the software development lifecycle: Testing. Imagine building a high-speed sports car, but forgetting to test the brakes before taking it onto the motorway! In software development, writing code is only half the job. We must make sure the system works correctly, securely, and actually does what the client asked for.
Don't worry if this topic feels broad at first. We will break it down step-by-step into clear, bite-sized sections with easy-to-remember analogies and memory aids.
1. The Foundation: Verification vs Validation
A classic exam question asks for the difference between Verification and Validation. While they sound almost identical, they focus on two different questions.
Verification ("Are we building the product right?")
• Definition: The process of checking whether the software conforms to its stated technical specifications and design rules.
• Focus: Did the developers follow the blueprints accurately? Is the code structured correctly?
• Techniques: Reviews, walkthroughs, inspections, and static code analysis.
Validation ("Are we building the right product?")
• Definition: The process of evaluating software during or at the end of development to determine whether it satisfies specified customer requirements and real-world user needs.
• Focus: Does the system actually solve the customer's problem in practice?
• Techniques: Executing test cases, user acceptance testing, alpha and beta testing.
Everyday Analogy: Imagine ordering a custom vegetarian pizza. If the chef prepares a vegetarian pizza with the exact toppings requested and cooks it at the right temperature, that is Verification (it meets the build specification). If you take a bite and realize it tastes awful or you were actually craving pasta all along, that is a failure of Validation (it did not satisfy the user's ultimate need).
Memory Trick:
• Veri-fact-ation: Checking the technical facts and specs.
• Vali-user: Making sure it brings value to the real user.
Key Takeaway: Verification checks adherence to the design specification (internal correctness), whereas Validation checks fulfillment of user needs (external fitness for purpose).
2. Testing Strategies: Black-Box vs White-Box Testing
When testing code, developers and testers look at the software from two distinct perspectives: from the outside looking in (Black-Box), and from the inside out (White-Box).
Black-Box Testing (Functional Testing)
• How it works: The tester does not look at or understand the internal code. They only have access to the inputs and expected outputs.
• Process: Test data is provided to the interface, and the output is checked against the functional specification.
• Example: Typing a username and password into a login screen to see if access is granted, without knowing the SQL queries or C# code running behind the button.
White-Box Testing (Structural or Glass-Box Testing)
• How it works: The tester has full access to the source code and internal architecture.
• Process: The tester designs tests to ensure that every path, branch, loop, and condition in the code executes correctly at least once.
• Example: Examining an if...else block in code and designing specific inputs so both the true branch and the false branch are executed and evaluated.
Quick Review:
• Black-Box: Focuses on what the software does (Interface & Outputs). Code is invisible.
• White-Box: Focuses on how the software does it (Logic & Structure). Code is visible.
3. Levels of Testing
Testing is not a one-time event at the end of a project. It happens in progressive levels, moving from small, isolated pieces of code up to the entire completed system.
Level 1: Unit Testing
• What it is: Testing individual components, subroutines, functions, or classes in complete isolation from the rest of the application.
• Who does it: The developer who wrote the code.
• Why: It is much easier to isolate and fix a bug in a 10-line function than in a 10,000-line completed system.
Level 2: Integration Testing
• What it is: Combining individual units or modules together and testing the communication, data flow, and interfaces between them.
• Why: Unit A might work perfectly on its own, and Unit B might work perfectly on its own, but errors can occur when Unit A passes parameters into Unit B.
Level 3: System Testing
• What it is: Testing the complete, fully integrated software system on target hardware to verify that all functional and non-functional requirements (such as performance, security, and recovery) are met.
• Who does it: Dedicated Quality Assurance (QA) testing teams.
Level 4: Acceptance Testing (Alpha & Beta Testing)
Acceptance testing is the final check before a system goes live, ensuring the client accepts the delivered product.
• Alpha Testing: Conducted in-house by internal testers or developers in a controlled environment, simulating real-world use to catch major bugs before external release.
• Beta Testing: The software is released to a selected group of real external end-users who test it in their own natural environments and report bugs or feedback back to the developers.
Did You Know? Popular video games and mobile apps frequently use open beta testing to discover how the software behaves across thousands of different phone models, screen sizes, and operating system versions!
4. Test Plans and Selecting Test Data
A Test Plan is a formal document that outlines the strategy, scope, resources, and schedule for testing activities. Central to any test plan are individual Test Cases.
Components of a Standard Test Case:
1. Test ID: A unique identifier (e.g., TC01).
2. Test Description: What is being tested (e.g., "Validate user age input field").
3. Test Data: The exact input value used.
4. Expected Result: What should happen if the system functions correctly.
5. Actual Result: What actually happened when the test was run.
6. Pass / Fail: An outcome indicator.
Categories of Test Data
To ensure thorough testing, examiners expect you to know and select three specific categories of test data. Let us use a rule where an acceptable exam mark must be an integer between \(0\) and \(100\) inclusive (\(0 \le \text{mark} \le 100\)).
• 1. Normal (Valid) Test Data:
Values that are typical, standard, and fall comfortably within the acceptable range.
Examples: \(45\), \(72\), \(88\). (Expected: Accepted).
• 2. Boundary / Extreme Test Data:
Values that sit right on the edge of acceptability, including values immediately at and just inside/outside the limits.
Examples: The boundary limits are \(0\) and \(100\). Extreme valid values are \(0\) and \(100\). Boundary invalid values are \(-1\) and \(101\). (Expected: \(0\) and \(100\) accepted; \(-1\) and \(101\) rejected).
• 3. Erroneous / Invalid Test Data:
Values that are completely outside the acceptable range or of the wrong data type altogether.
Examples: \(-50\), \(999\), or entering text like "Pass" into a numeric field. (Expected: Rejected with a helpful error message).
Common Mistake to Avoid: When asked for boundary data for a range like \(1\) to \(10\), do not give middle values like \(5\). You must provide the edge limits (\(1\) and \(10\)) and the numbers directly adjacent to them (\(0\) and \(11\)).
5. Static Testing: Dry Running and Trace Tables
Testing does not always require running the program on a computer. Static testing involves analyzing and evaluating code without executing it.
Dry Running
• A manual, pen-and-paper exercise where the programmer works through the logic of an algorithm or code snippet step-by-step using a set of sample inputs.
• Helps spot logic errors (such as off-by-one errors or infinite loops) early in development.
Trace Tables
• A structured grid used during a dry run to record the values of variables, condition outcomes, and outputs as they change line-by-line through the execution of an algorithm.
• Columns represent individual variables and conditions; rows represent each step or loop iteration.
Walkthroughs and Formal Code Inspections
• Structured Walkthrough: The author of the code leads a peer group through the source code line-by-line to gather feedback and spot oversights.
• Code Inspection: A formal, rigorous peer review led by a trained moderator using checklists to identify defects against established coding standards.
Key Takeaway: Dry runs and trace tables allow developers to trace logic errors systematically before code is even compiled or executed.
6. Summary & Quick Reference Checklist
Before moving on to the next chapter, check if you can confidently answer the following:
• Can you explain the difference between Verification (meets specs) and Validation (meets user needs)?
• Can you contrast Black-Box (functional, code hidden) with White-Box (structural, code visible) testing?
• Can you list the 4 main levels of testing in order: Unit \(\rightarrow\) Integration \(\rightarrow\) System \(\rightarrow\) Acceptance (Alpha/Beta)?
• Can you select Normal, Boundary, and Erroneous test data for a given scenario?
• Can you describe how a Trace Table and Dry Run help uncover logic bugs?