Chapter: Evaluating the Solution
Welcome to the final, crucial stage of the software development process! You have designed, coded, and tested your software. Now it is time to take a step back and ask: "How well does this system actually work, and how successful was the project?"
Don't worry if evaluation feels slightly daunting or subjective at first. In CCEA A2 Software Systems Development, evaluating a solution is not about claiming your software is 100% perfect. In fact, examiners award the highest marks to students who demonstrate honest, critical, and realistic insight into their work!
Why is Evaluation Important?
Imagine a car manufacturer building a brand-new electric vehicle. Before selling it to the public, engineers must review whether it meets safety regulations, check crash test data, identify battery range limitations, and plan features for next year's model. Software evaluation does the exact same thing for your digital solutions.
In this chapter, you will learn how to:
• Evaluate a completed system against original user requirements and objectives.
• Use test plan results as concrete evidence of software success.
• Identify realistic system limitations and constraints.
• Propose sensible future enhancements.
• Conduct a thorough self-evaluation of your own performance and project management.
1. Evaluating Against User Requirements and System Objectives
The primary measure of any software solution's success is whether it satisfies the needs of the client and end-users identified during the initial analysis phase.
Functional vs. Non-Functional Evaluation
When reviewing requirements, you must evaluate two distinct categories:
• Functional Requirements: These describe what the system does (specific features and behaviors). For example: "The system must allow a user to add a new appointment, update customer records, and calculate total invoices." In your evaluation, you systematically confirm whether each feature was fully implemented, partially implemented, or omitted.
• Non-Functional Requirements: These describe how the system performs (quality attributes). Examples include usability, response times, data security, and system reliability. For instance: "Does the login screen authenticate users in under \(2\) seconds?" or "Is user data securely hashed in the database?"
Technique: The Requirements Traceability Approach
To avoid vague statements, link every evaluation comment directly back to the original requirements specification. A simple method is checking each requirement with a clear outcome:
• Met: The feature works exactly as planned and meets all acceptance criteria.
• Partially Met: The feature exists, but with minor constraints (e.g., the user can view an invoice, but cannot export it to PDF).
• Unmet: The feature was not delivered due to time constraints, technical hurdles, or scope changes.
Quick Key Takeaway: Never just say "The program works well." Always point to specific functional and non-functional requirements and explain with evidence how each was achieved.
---2. Evaluating Against Test Plan Results
Your testing phase provides the hard, factual evidence needed to evaluate your system. Without test data, an evaluation is simply an opinion.
Using Test Logs as Evidence
During testing, you executed tests using normal, boundary/extreme, and erroneous/invalid data. In your evaluation, you reflect on those results:
• Expected vs. Actual Outcomes: Highlight tests where the software behaved exactly as planned. This proves that core calculations, data validation, and database updates work correctly.
• Discrepancies and Bug Fixes: Discuss tests that initially failed, what the root cause was (e.g., an off-by-one loop error or unhandled null exception), and how the issue was resolved.
• Robustness and Error Handling: Evaluate how well your program prevents crashes when unexpected input occurs (for example, entering letters into a numeric telephone field).
User Acceptance Testing (UAT)
Did the target audience or client test the system? Evaluating client feedback provides direct insight into user interface (UI) navigation, workflow efficiency, and overall user experience (UX).
Did You Know? In professional software engineering, finding bugs during testing is considered a huge success, not a failure! A comprehensive test log that uncovers errors and demonstrates fixes proves rigorous quality assurance.
Quick Key Takeaway: Test results are your proof of software quality. Use specific test outcomes to justify your conclusions about reliability and robustness.
---3. Identifying Limitations of the Solution
Every software system in the world has limitations—from single-developer projects to operating systems built by global tech giants. Identifying what your software cannot do demonstrates mature, analytical thinking.
Common Types of System Limitations
• Scalability Constraints: The application might handle \(100\) database records smoothly, but query performance may degrade if the dataset grows to \(100{,}000\) records.
• Platform / Environment Dependency: The application might only run on Windows desktop environments with a specific version of the .NET runtime installed, lacking cross-platform or mobile accessibility.
• Hardware / Network Dependencies: The system might require a continuous local area network connection to access the database, offering no offline caching or data sync.
• Security Constraints: Basic authentication might be functional, but the system may lack advanced security mechanisms such as Two-Factor Authentication (2FA) or role-based access control (RBAC).
• Edge Cases: Certain complex business scenarios (e.g., processing refunds across multiple payment methods) might not be automated and require manual intervention.
Crucial Distinction: Limitation vs. Unfixed Bug
• A bug is an error in code that causes it to behave incorrectly against its design (e.g., calculating a \(20\%\) tax rate as \(0.02\) instead of \(0.20\)).
• A limitation is an intentional or structural boundary of the system's current design (e.g., the system only supports payments in GBP (\pounds) and does not handle foreign currency exchange rates).
Quick Key Takeaway: Examiners look for critical self-awareness. Be honest and clear about system limitations—it shows technical maturity.
---4. Potential for Future Enhancements
A good developer always looks ahead. Future enhancements describe how the software could realistically be expanded, updated, or improved in version \(2.0\).
Linking Enhancements to Limitations
The best recommendations for future development directly address the limitations you identified earlier:
• Limitation: The system only operates on desktop PCs.
Enhancement: Develop a responsive web portal or mobile companion app using a REST API to allow staff to access data on the road.
• Limitation: Reports must be generated manually by selecting dates.
Enhancement: Implement an automated background service that emails weekly summary reports and PDF charts directly to managers.
• Limitation: Single-tier database access on a local drive.
Enhancement: Migrate the database to a secure cloud platform (such as Microsoft Azure SQL) to ensure automated backups, high availability, and remote multi-user access.
Memory Aid: The "S.M.A.R.T." Enhancement Rule
Make sure your suggested enhancements are:
• Specific: Clearly state the feature (e.g., "Implement automated SMS appointment reminders").
• Meaningful: Explain the business value to the client.
• Achievable: Propose realistic technical upgrades rather than vague science fiction concepts.
Quick Key Takeaway: Future enhancements should be logical, realistic progressions from your current solution that directly solve existing limitations.
---5. Evaluation of Own Performance
The final component of evaluating the solution is reflecting on your personal performance as a developer and project manager throughout the software development life cycle (SDLC).
Key Areas of Self-Evaluation
• Time Management and Planning: Compare your initial project plan (such as your Gantt chart milestones) against your actual progress. Did certain phases (like debugging database relations) take longer than expected? How did you adapt your schedule to stay on track?
• Technical Skill Development: Reflect on new programming constructs, object-oriented concepts, or database query techniques (such as complex SQL joins or LINQ queries) that you learned and applied during the project.
• Problem-Solving and Resilience: What were the biggest technical roadblocks you encountered? How did you research and resolve them (e.g., consulting documentation, refactoring code, step-by-step breakpoint debugging)?
• Lessons Learned / What Would You Do Differently: If you were to start the project again from scratch, what would you change? (e.g., "I would spend more time normalizing database tables in the design stage to avoid complex refactoring during implementation.")
Summary Checklist: The Complete Evaluation Framework
When writing your final evaluation, ensure you have covered all five pillars:
1. User Requirements: Did the system deliver what was promised (both functional and non-functional)?
2. Test Evidence: How do the actual test results prove reliability, robustness, and accuracy?
3. Limitations: What are the real-world boundaries, constraints, and scope cut-offs of the system?
4. Future Enhancements: What logical features or architectural upgrades should be included in future versions?
5. Individual Performance: How effectively did you manage time, overcome technical challenges, and develop your skills?
Common Mistakes to Avoid:
• Mistake 1: Claiming the system has zero limitations. (Always identify genuine constraints).
• Mistake 2: Listing future enhancements that were actually mandatory core requirements that you simply ran out of time to build.
• Mistake 3: Writing vague self-reflection without referencing concrete problems solved or specific project milestones.