Methodological issues: reliability, validity and ethics

Introduction: The "Quality Control" of Psychology

Welcome! When psychologists conduct research, they aren't just looking for interesting stories; they are trying to find scientific truths. But how do we know if a study is actually "good"? How do we know we can trust the results?

In this chapter, we look at the three pillars of high-quality research: Reliability (consistency), Validity (accuracy), and Ethics (morality). Think of these as the "quality control" checks that every piece of psychological research must pass. Whether you are aiming for a top grade or just trying to get your head around the basics, mastering these terms is essential for Component 01.

1. Reliability: The Quest for Consistency

Reliability is all about consistency. If you stepped on a bathroom scale and it said 70kg, then stepped off and back on again and it said 85kg, you wouldn't trust it. That scale has low reliability. In psychology, if a test or observation is repeated, it should produce the same results to be considered reliable.

Types of Reliability

• Internal Reliability: This is the extent to which a measure is consistent within itself. For example, if a person takes a personality test for "Extraversion," all the questions should be measuring the same thing.
• External Reliability: This is the extent to which a measure varies from one use to another. If you take the same test two weeks apart, do you get a similar score?

How do we measure Reliability?

Psychologists use specific "checks" to ensure their research is consistent:

1. Test-Retest: Giving the same participants the same test on two different occasions. If the scores are similar, the test is reliable.

2. Split-Half: Splitting a test in two (e.g., comparing scores on the first 10 questions to the last 10 questions). If the scores are similar, the test has internal reliability.

3. Inter-rater Reliability: This is vital for observations. It is the degree of agreement between two or more researchers. If two people watch the same child and both record 5 "aggressive acts," they have high inter-rater reliability.

Quick Review: Reliability = Consistency. If you see the word "consistent" or "replicable" in an exam question, think Reliability!

2. Validity: The Quest for Truth

Validity is about accuracy. Does the research actually measure what it claims to measure? A scale might be reliable (it gives you 80kg every single time), but if you actually weigh 70kg, the scale is invalid. It is consistent, but it is wrong.

Internal Validity (Inside the study)

Does the study test what it intended to? Are the results caused by the Independent Variable and not something else?
Types include:

• Face Validity: Does the test look like it measures what it's supposed to "on the face of it"?
• Construct Validity: Does the test actually measure the theoretical "construct" (like intelligence or love)?
• Concurrent Validity: Do the results match a different, already established test of the same thing?
• Criterion Validity: How well does the measure predict future behavior?

External Validity (Outside the study)

Can we apply these findings to the real world?
Types include:

• Ecological Validity: Can the results be generalized to real-life settings? (Laboratory experiments often struggle with this).
• Population Validity: Can the results be generalized to other groups of people beyond the sample used?

Threats to Validity

Sometimes, participants or researchers accidentally "break" the validity of a study:

Demand Characteristics: When participants figure out the aim of the study and change their behavior to "fit in" or help the researcher.

Social Desirability: When participants answer questions in a way that makes them look "good" or "normal" rather than being honest.

Researcher/Observer Bias: When a researcher’s own expectations influence how they record data (e.g., seeing "aggression" because they expect to see it).

Researcher/Observer Effects: When the physical presence of the researcher changes the way participants behave (also known as the Hawthorne Effect).

Key Takeaway: A study can be reliable but not valid, but it cannot be truly valid if it isn't reliable!

3. Representativeness and Generalisability

These two terms are linked to Population Validity:

Representativeness: Does the sample of participants reflect the target population? If you only study university students, your sample isn't representative of the elderly.

Generalisability: This is the outcome. If your sample is representative, you can generalise (apply) your findings to the wider world.

4. Ethical Considerations: The BPS Code

In the UK, the British Psychological Society (BPS) sets the rules for how to treat participants. These are organized into four main pillars. Don't worry if this seems like a lot to memorize—just remember the "Big Four" headings!

I. Respect

• Informed Consent: Participants should know what they are getting into before they agree to take part.
• Right to Withdraw: Participants must be told they can leave at any time, and they can also withdraw their data after the study is finished.
• Confidentiality: Personal data (names, addresses) must be kept private.

II. Competence

Researchers must work within their own limits. They shouldn't try to give medical advice if they aren't qualified to do so.

III. Responsibility

• Protection of Participants: Participants should leave the study in the same physical and mental state as they entered. They must be protected from harm and distress.
• Debrief: After the study, the researcher must explain the true aim, check the participant's well-being, and answer any questions.

IV. Integrity

• Deception: Researchers should avoid lying to participants. If they must use deception (to avoid demand characteristics), it must be justified, and the participants must be debriefed as soon as possible.

Did you know? Some of the core studies you'll learn about in Component 02, like Milgram's obedience study, are famous because they are so controversial regarding these ethical rules!

Quick Review Table

Use this table to keep the most commonly confused terms separate:

For more information on the specific methods used to collect data (like observations or experiments), see the chapter on "Research methods and techniques".