Introduction to Big Data and Data Science

Welcome to one of the most exciting parts of the APM syllabus! In the past, performance management relied on old financial reports that told us what happened last month. Today, Big Data and Data Science allow organizations to see what is happening right now and even predict what will happen tomorrow. For an ACCA student, understanding these methods is key because you will often be asked to advise a company on how to use technology to gain a competitive advantage.

In this chapter, we focus on what Big Data actually is, the step-by-step process of handling data, the different levels of analytics, and the various types of data we can now analyze.

1. Understanding Big Data

Big Data refers to datasets that are so large and complex that traditional data processing software just can't handle them. Think of the difference between a small grocery store's paper ledger and the billions of transactions processed by Amazon every day.

The 4 Vs of Big Data

To help you remember the characteristics of Big Data, think of the 4 Vs:

  • Volume: The sheer amount of data. We are talking about terabytes and petabytes of information.
  • Velocity: The speed at which data is generated and processed. For example, social media feeds or credit card transactions happen in real-time.
  • Variety: Data comes in many forms—not just numbers in a spreadsheet, but photos, videos, and social media posts.
  • Veracity: The "truthfulness" or quality of the data. Is the data accurate and can we trust it to make decisions?

Impact on Performance Management: Big Data allows for real-time monitoring. Instead of waiting for a month-end variance report, a manager can see a drop in production efficiency the moment it happens and fix it immediately.

Quick Tip: In the exam, if you are asked about the "challenges" of Big Data, Veracity is often the best one to discuss. If the data is messy or biased, the performance reports will be wrong!

2. The Data Science Process

Data science isn't just magic; it follows a logical, step-by-step method. You need to understand this flow to evaluate how an organization manages its insights.

Step 1: Setting the Goals

Before looking at data, you must know what problem you are trying to solve. Example: "We want to reduce customer churn by 10%."

Step 2: Selecting the Data

Deciding which data sources are relevant. This could be internal data (sales records) or external data (weather patterns or competitor prices).

Step 3: Cleaning the Data

This is often the most time-consuming part. "Dirty" data (duplicates, errors, or missing values) must be removed. If you skip this, your results will be "Garbage In, Garbage Out".

Step 4: Transforming the Data

Converting data into a usable format. For example, changing raw dates into "day of the week" to see if sales are higher on Saturdays.

Step 5: Storing the Data

Data must be stored securely and in a way that it can be easily accessed for analysis, often using cloud technology or data warehouses (which are covered in more detail in the next chapter).

Key Takeaway: Performance management is only as good as the data behind it. Skipping the "cleaning" or "goal-setting" stages leads to expensive mistakes.

3. The Four Levels of Analytics

In APM, you must be able to apply and evaluate these four types of analytics. They move from simple "looking back" to complex "looking forward."

A. Descriptive Analytics ("What happened?")

This summarizes past data. Example: A report showing that sales fell by \(5\%\) last quarter. Most traditional management accounts are descriptive.

B. Diagnostic Analytics ("Why did it happen?")

This digs deeper to find patterns or causes. Example: Noticing that sales fell because a specific shipping route was blocked. It looks for correlations.

C. Predictive Analytics ("What will happen?")

Using historical data to forecast the future. Example: Using past seasonal trends to predict that the company will need \(20\%\) more inventory in December. (Note: Regression analysis is a common tool used here).

D. Prescriptive Analytics ("How can we make it happen?")

The most advanced level. It suggests a course of action. Example: An algorithm that automatically adjusts prices every hour to maximize total profit based on current demand.

Analogy: Imagine you are driving a car. Descriptive is looking at the speedometer. Diagnostic is seeing the "low oil" light and knowing why the engine is hot. Predictive is seeing a storm cloud ahead and knowing the road will be slippery. Prescriptive is the GPS telling you exactly which turn to take to avoid the storm and arrive on time.

4. Data Types for Analytics

Modern performance management doesn't just look at numbers (\(1, 2, 3\)). We now use unstructured data to gain insights:

  • Text: Analyzing customer reviews or emails to measure "sentiment." If customers use words like "slow" or "disappointed," the Net Promoter Score (NPS) is likely to drop.
  • Image: Using photos to check product quality on an assembly line or monitoring footfall in a retail store.
  • Video: Security footage can be analyzed to see how long customers wait in a queue, which is a key non-financial performance indicator.
  • Voice: Analyzing recordings of customer service calls to see if staff are following scripts or if customers sound frustrated.

5. Risks and Challenges

While Big Data sounds perfect, it comes with significant hurdles that you should mention in your exam answers:

  • Cost: Buying the technology and hiring data scientists is very expensive.
  • Skill Gap: Existing accountants may not have the skills to interpret complex data models.
  • Data Overload: Managers can get overwhelmed by too much information (often called "analysis paralysis"), making it harder to focus on Critical Success Factors (CSFs).
  • Security and Ethics: Storing massive amounts of personal data creates a risk of cyber-attacks and ethical dilemmas regarding privacy.

Quick Review Quiz

Q: Which 'V' refers to the speed of data?
A: Velocity.

Q: Which type of analytics tells a manager what to do next?
A: Prescriptive Analytics.

Q: Why is "Cleaning" data important?
A: To ensure accuracy and avoid making decisions based on "garbage" data (Veracity).

Cross-reference: To see how these methods are used with specific technologies like AI or Regression, please refer to the later chapters in Section D.