Welcome to System Evaluation!
Imagine you have just designed, built, and launched a brand-new website for a local sports club or created an interactive educational game. How do you know if it is actually any good? Does it do what the client asked for? Can real people use it easily, or does it crash every time someone clicks a button?
This is where system evaluation comes in! In Unit 2: Digital Authoring Concepts (Section 2.7), evaluation is all about taking a step back to critically review, verify, and appraise a finished digital product. Whether you are answering questions in the 1 hour 30 minute written exam (worth 40% of your GCSE) or working on your Unit 3 Controlled Assessment, mastering evaluation will help you gain top marks!
---1. What is System Evaluation? (Purpose & Principles)
System Evaluation is the formal, critical assessment of a completed digitally authored system against predefined criteria (such as original user requirements and design specifications) to determine its overall effectiveness, quality, and fitness for purpose.
Don't worry if this sounds a bit technical! Think of it like taking a freshly built car out on a test track with a checklist: you are checking if the steering works, if the seats are comfortable, if it matches what the buyer ordered, and if anything needs fixing before it goes on the road.
Core Objectives of System Evaluation
When evaluating any digital product, a developer must achieve four main goals:
1. Verify User Requirements: Check whether the finished system meets all the goals laid out in the initial client brief and user requirement specification.
2. Measure Performance & Usability: Test how fast, stable, and easy to use the system is under real-world target operating conditions.
3. Identify Issues: Spot any bugs, performance bottlenecks, functional discrepancies, or accessibility barriers that slipped through.
4. Recommend Improvements: Suggest realistic enhancements, maintenance schedules, or future upgrades for the next version.
Quick Summary: Evaluation asks three simple questions: Did we build what was asked for? Does it work smoothly for the target audience? How can we make it even better?
---2. The Four Key Evaluation Dimensions
To evaluate a digital system thoroughly, you need to look at it from multiple angles. CCEA requires you to examine four key dimensions:
Dimension 1: Fitness for Purpose & Meeting User Requirements
A digital system is fit for purpose if it successfully does the job it was created to do. To check this, you must systematically go through each original requirement from the client's specification and classify it:
• Fully met: The feature is completely implemented, works without errors, and fulfills the brief (e.g., "A user can register an account, and their details are saved accurately into the database").
• Partially met: The feature works to some extent, but lacks key functionality or polish (e.g., "A user can register, but the system does not send a confirmation email").
• Not met: The feature was not implemented at all, or fails completely during operation.
Top Tip: In exam questions, always provide justification and evidence for why a requirement is fully, partially, or not met!
Dimension 2: User Interface (UI) and User Experience (UX)
The interface is how users interact with your system. A great interface makes a system feel intuitive and enjoyable.
• Navigation Structures: Is the system easy to move around? Does it use a linear structure (step-by-step, like a checkout process), a hierarchical structure (top-level homepage branching into sub-pages), or a non-linear / mesh network (freely moving between pages)?
• Navigational Aids: Are clear tools provided, such as drop-down menus, breadcrumb trails, search bars, and interactive navigation buttons?
• GUI & WIMP Conventions: Does the interface follow standard Graphical User Interface rules using WIMP (Windows, Icons, Menus, Pointers)? Following familiar conventions helps users know what to click without getting confused.
• Visuals & Typography: Is text easy to read? Is there strong colour contrast between text and background? Are font styles and sizes suitable for the target audience?
• Accessibility & Target Audience: Is the design appropriate for its intended audience? (e.g., large buttons and bright primary colours for young children, or clean layouts with high-contrast text and accessibility aids for older or visually impaired users).
Dimension 3: Performance and Technical Robustness
A system can look beautiful, but if it is slow, broken, or insecure, it fails as a digital solution.
• Loading Times & Asset Optimisation: Are multimedia files (images, audio, video) properly compressed and web-optimised? High-resolution bitmap images that have not been compressed cause huge file sizes, leading to heavy data transfer overheads and slow page loading speeds.
• Cross-Platform Compatibility & Responsiveness: Does the system display and function correctly across different web browsers (e.g., Chrome, Edge, Safari), various screen resolutions, and different devices (smartphones, tablets, desktop PCs)?
• Data Validation and Integrity: If the system connects to a database or uses input forms, do the validation rules work? These include:
- Presence checks: Ensuring required fields are not left blank.
- Range checks: Ensuring numbers fall within set boundaries (e.g., an age between \(1\) and \(120\)).
- Length checks: Ensuring data has an acceptable number of characters.
- Format checks: Ensuring data follows a set pattern (e.g., a postcode or email format).
Dimension 4: Maintenance and Scalability
Digital systems are rarely static; they need updates over time.
• Code & Structure Maintainability: Are embedded scripts, macros, and authored structures clearly organised, well-commented, and easy for another developer to understand and edit?
• Content Scalability: How easy is it to add new products, update dynamic content, add assets to multimedia libraries, or update database records without breaking the layout?
Key Takeaway: When evaluating a system, always cover (1) User Requirements, (2) UI/UX Design, (3) Technical Performance, and (4) Maintenance/Scalability.
---3. Feedback Collection and Evaluation Methods
To evaluate a system objectively, you cannot rely only on your own guesses. You must gather reliable evidence using formal data collection methods.
A. User and Client Feedback
• Questionnaires / Surveys: These can be distributed to a representative sample of target users. They use a mix of closed questions (e.g., rating scales from \(1\) to \(5\) for quantitative data) and open-ended questions (e.g., text boxes for qualitative feedback and specific user opinions).
• Client Sign-off Meetings / Interviews: Direct structured or semi-structured meetings between the developer and the client. They go through the original requirements specification item-by-item to ensure the client is satisfied before formal sign-off.
• User Testing & Observation Sessions: Watching target users perform specific, realistic tasks (such as finding an item, submitting a contact form, or running a search query). This allows developers to observe real friction points, confusion, and interface bottlenecks in real time.
B. Review of Test Plans and Test Results
Evaluation requires looking back at your formal test tables to see how the system behaved when fed different types of data:
• Normal (Valid) Data: Standard data that the system expects (e.g., entering the word "Smith" into a surname field).
• Extreme (Boundary) Data: Data at the outer limits of acceptable ranges (e.g., entering exactly \(1\) or \(100\) if the allowed range is \(1\) to \(100\)).
• Invalid (Erroneous) Data: Data that is unacceptable or of the wrong type (e.g., entering letters into a phone number field or leaving a compulsory field blank).
By comparing expected results with actual results, the evaluation notes which defects were resolved during development and which remain as system limitations.
C. Self-Evaluation and Critical Reflection
The developer must also reflect honestly on their own process:
• Did I choose the best authoring software and multimedia tools for this project?
• Was my time management and development workflow efficient?
• Which design decisions worked well, and what would I do differently next time?
4. Formulating Future Improvements and Next Steps
A complete evaluation does not stop at pointing out flaws—it must provide clear, actionable next steps for future development:
• Remediating Errors: Fixing any partially met requirements, broken hyperlinks, validation glitches, or unhandled errors identified during user testing.
• Adapting for Future Audience Needs: Expanding the functional scope of the system (e.g., adding automated email confirmations, implementing mobile-responsive stylesheets, or building advanced database search queries).
5. Common Pitfalls to Avoid in the Exam
Examiners frequently report the same mistakes year after year. Watch out for these traps:
• Avoid Vague & Subjective Comments: Never just write "The website looks nice and is easy to use." This earns zero marks! Instead, write: "The high colour contrast between the dark blue text and white background ensures clear readability for visually impaired users, meeting the accessibility requirement."
• Don't Confuse Testing with Evaluation: Testing is running the checks (e.g., typing invalid data into a box). Evaluation is analysing what the results mean for the overall quality and fitness for purpose of the system.
• Always Link to the Given Scenario / Audience: If the exam scenario is about an ordering system for a local bakery or a learning app for primary school children, your evaluation points and UI comments must directly reference that specific audience.
• Be Honest About Weaknesses: High-scoring evaluation answers (AO3) are balanced. Don't pretend a system is 100% perfect if the scenario describes bugs or missing features. Acknowledge limitations and explain how to fix them!
• Give Technical, Contextual Improvements: Instead of saying "add more graphics," suggest: "Compress all bitmap images to web-optimised formats to reduce file sizes, decrease bandwidth usage, and improve page loading times."
Quick Revision Checklist
Before sitting your exam, make sure you can answer these questions with confidence:
✔ What is the definition and main purpose of system evaluation?
✔ Can you describe the four dimensions of evaluation (Fitness for Purpose, UI/UX, Performance, and Maintenance)?
✔ What is the difference between fully met, partially met, and not met requirements?
✔ What methods are used to collect user feedback (questionnaires, interviews, observation)?
✔ How do you use test plans with normal, extreme, and invalid data to evaluate a system?
✔ Can you propose specific, technical improvements to resolve system limitations?