Introduction to Testing and Evaluation in AS 2
Welcome to the final stage of your AS 2: Coursework – Product Development folder! After researching, redesigning, and manufacturing your 3D prototype, it is time to put your hard work to the test. Testing and evaluation is not just about showing that your product works; it is about critically judging how well it solves the problems you identified in the original product and deciding how it could be manufactured commercially.
Don't worry if your prototype didn't turn out 100% perfect. In Technology and Design, finding a flaw and explaining why it happened and how to fix it earns far higher marks than pretending everything is flawless!
Quick Unit AS 2 Snapshot:
• Assessment Weighting: 50% of your AS qualification (20% of the overall A Level).
• Portfolio Length: 10 to 12 A3 sheets (or equivalent e-portfolio) plus your practical 3D model.
• Testing & Evaluation Section: Typically covers 1 to 2 A3 pages of your portfolio.
1. Testing Against the Redesign Specification
In Unit AS 2, you are re-engineering an existing product rather than designing completely from scratch. Therefore, your tests must measure whether your redesign successfully solved the shortcomings of the original unimproved product.
You must formulate objective, measurable, and user-centred test procedures that check your prototype against your original Redesign Specification across five core areas:
1. Function and Purpose
Does the re-engineered mechanism or component operate smoothly? Did it achieve the intended functional improvement? (e.g., Does a redesigned latch release with smooth mechanical movement without jamming?)
2. Fitness for Purpose
How effectively does the finished prototype solve the exact faults found during your initial product disassembly and analysis? Does it perform its day-to-day job effectively?
3. Ergonomic Suitability
How does the user interact with the product? Test the grip, comfort, anthropometric dimensions, and ease of physical operation across your target demographic.
4. Aesthetics and Form
Critique the visual proportion, styling, colour consistency, and surface finish. Does the prototype match the stylistic preferences of the intended market?
5. Safety and Durability
Are there any pinch points, sharp edges, or structural weaknesses? Does the product withstand repeated operational cycles, or do components show signs of premature wear?
Memory Aid (F-F-E-A-S): Remember the 5 pillars of testing with "Fast Findings Enable Accurate Solutions" (Function, Fitness, Ergonomics, Aesthetics, Safety).
Key Takeaway: Never write a generic test. Every single test must directly link back to an explicit requirement from your Redesign Specification.
---2. Gathering Evidence: Quantitative and Qualitative Testing
To evaluate thoroughly, you need both hard numerical data and user feedback. Relying on only one type of data weakens your evaluation.
A. Quantitative Testing (Numerical Data)
Quantitative tests produce empirical, measurable numbers. They leave no room for opinion.
• Dimensional Accuracy: Measuring parts with digital callipers to check if manufactured sizes match technical drawings (e.g., \( \text{Tolerance} = \pm 0.5\text{ mm} \)).
• Mass / Weight Comparison: Weighing the prototype on digital scales to compare it with the original product.
• Load-Bearing Capacity: Measuring how much weight or force the redesigned part can take before deflecting.
• Cycle Times & Speed: Timing how long an action takes to complete (e.g., time taken to fold, assemble, or actuate a mechanism in seconds).
B. Qualitative Testing (User Feedback & Observation)
Qualitative tests focus on user experience, opinions, and descriptive observations.
• User Group Questionnaires: Giving structured surveys with rating scales to people in your target market.
• Observation Logs: Watching a third party interact with your prototype without prompting them, noting any confusion or physical struggle.
• Comparison Matrices: Setting up a side-by-side comparison table directly comparing your redesigned prototype against the original unimproved product.
C. Photographic Evidence in Context
Examiners require clear photographic evidence showing your prototype being tested in a realistic working environment by a member of the target user group. A photo of a prototype sitting on a clean school workbench does not prove that it works in the real world!
Key Takeaway: High-scoring folders combine hard numbers (quantitative) with real target-user feedback (qualitative) supported by clear photographic proof.
---3. Critical Evaluation: Analysis Over Description
One of the most common mistakes in AS 2 is writing a story of what you did instead of an evaluation of how well it performed.
The Difference Between Description and Critical Analysis
• Descriptive (Low Marks): "I tested the handle by pulling it. It worked well and felt nice to hold. The user liked the colour."
• Evaluative & Analytical (High Marks): "Under a quantitative load test of \( 50\text{ N} \), the redesigned handle deflected by \( 1.2\text{ mm} \). The 5mm radius fillets prevented stress concentrations, eliminating the fracture point identified in the original product. However, user feedback noted that the smooth acrylic surface lacked grip when wet, showing that a textured elastomer overmould is required for true ergonomic suitability."
Explaining the 'Why'
When evaluating results, always explain why a feature succeeded or failed by referencing:
• Material Properties: Did the material have suitable tensile strength, rigidity, or elasticity?
• Fabrication Techniques: Did 3D printing layer orientation cause a weak shear plane? Did laser cutter kerf affect joint tightness?
• Tolerances: Were the mechanical clearances between moving parts too tight or too loose?
Key Takeaway: Don't just say what happened. Explain why it happened using materials science, joining methods, and mechanical tolerances.
---4. Proposals for Modification and Commercial Manufacture
A school workshop prototype is rarely identical to a final mass-produced item. The final part of your AS 2 evaluation requires you to propose realistic future modifications and explain how the product would be adapted for industrial manufacturing.
A. Design Refinements and Modifications
Based on your test results, outline clear, annotated sketches or CAD snippets showing modifications. These must be justified directly by your test data, not invented out of thin air.
B. Moving from One-Off Workshop to Commercial Production
You must explain how manufacturing methods would change when scaling up from a one-off prototype to batch or mass production:
• Workshop Prototype: 3D printed PLA casing or fabricated sheet acrylic.
• Commercial Mass Production: High-volume Injection Moulding using ABS or Polypropylene (incorporating draft angles, uniform wall thicknesses, and internal strengthening ribs).
• Workshop Prototype: Hand-drilled or milled aluminium bracket.
• Commercial Mass Production: Die Casting or progressive Stamped Sheet Tooling for rapid, repeatable batch production.
Key Takeaway: Commercial proposals show the examiner that you understand the realities of industrial manufacturing beyond the school workshop.
---5. Examiner Pitfalls to Avoid
CCEA examiners regularly highlight the following weaknesses in AS 2 Testing and Evaluation sections. Review this list to ensure your work hits the top mark bands!
1. Descriptive Storytelling: Avoid writing a step-by-step diary of your testing day. Keep your commentary strictly analytical and focused on performance criteria.
2. Testing Only on Yourself: As the designer, you know how your prototype works. You must have independent third-party users or members of the target demographic test it.
3. Superficial "Tick-Box" Evaluations: Stating "The specification asked for it to look modern, and it looks modern so this point is met" scores poorly. Back up your claims with user ratings, aesthetic feedback, or dimensional checks.
4. Ignoring Failures and Breakages: Hiding flaws loses marks. Identifying a failure, analysing why the material or joint failed, and suggesting a correct engineering fix gains maximum marks.
5. Unrealistic Modifications: Proposing impossible changes or ignoring industrial manufacturing constraints shows a lack of technical understanding.
6. Summary and Final Checklist
Before submitting your Testing and Evaluation pages (1–2 A3 sheets), check that you have included:
• Clear tests checking all aspects of the Redesign Specification.
• Quantitative data (measurements, tolerances, weights, cycle times).
• Qualitative feedback (target user surveys, observation notes, comparison matrix).
• Photographs showing the prototype being tested in a realistic environment by users.
• Detailed analysis explaining why components succeeded or failed (referencing materials and tolerances).
• Justified future modifications based directly on your testing evidence.
• Specific details on how the design would transition to commercial manufacturing (e.g., injection moulding, die casting).