Thinking critically about research

Stage 1, Week 5 tutorial.

stage1

tutorial

Tutor tasks

Log this tutorial in S4

Did your tutees miss the tutorial? If so, use this link to send a pre-written email:

Send email to missing students

Session overview

This session gets students thinking critically about research.

Task 1: Discuss a mock news item about a correlational study
Task 2: Interpret experimental data and draw valid conclusions
Task 3: Distinguish weak from strong critiques, and practice writing limitations

Aim to get through the tasks 1 and 2 by roughly half way through the session.

Why these tasks matter:

Core critical thinking skills for PSYC421 research reports (especially the discussion section)
Practice interpreting figures (they recently learned about distributions)
Learn to write specific, thoughtful limitations rather than generic critiques

Notes:

Hypothesis testing is introduced in week 5 (students may or may not have covered it yet).
Correlations have not yet been taught (PSYC424, Semester 2), but should not be an obstacle for Task 1

Task 1: Pop science

Topic: Reading a mock news item reporting a new study correlating chocolate consumption and happiness.

Materials

Chocolate and happiness study (handout or QR code below)

Procedure

Show students the QR code

This links to a ‘news’ item which they can open on their phones
Read the news item aloud to the group.

Group discussion:

Would they believe this story? Why?
Does it matter that it’s about chocolate?
Do their prior beliefs play a role? What if it was about washing dishes?

In small groups (2-3 students):

Read the method and results sections.
Students to discuss:
- What are the limitations of this design?
- What external factors could be confounded (i.e. influence both chocolate consumption and happiness)?
- How would you design a study to show chocolate eating causes happiness?

Whole group discussion:

Share ideas from the small groups
Did the study details change their views?
Lead discussion toward experimental design (chocolate as an independent variable)
This sets up Task 2: interpreting experimental data

Task 2: Drawing valid conclusions from experiments

Goal: Move beyond ‘A-level’ critiques of evidence (e.g., “study shows \(x\) but it was done in 1972 so we should ignore it”, or “but these were just Plymouth students…”)

Key distinction:

What was observed (the effect exists), vs.
Why it happened (the explanation)

Notes:

Students often:

Reject effects because they can think of alternative explanations
Accept effects because they seem plausible (even when evidence doesn’t support it)

Part 1: What influences memory?

Ask students: Which of the following statements do they agree with?

All other things being equal, how much a person is likely to remember about a recent event is influenced by:

How old that person is

How drunk that person is

How much attention the person was paying

The learning style that person has

How unusual or important the event is

Ask: What other factors might influence memory?

Generate 1-2 ideas in pairs, then share as a group

Agree with them! Generally you should agree with the students: many things can influence memory.

Next: Show Scenario 1.

Part 2: Scenarios

Materials

Memory study scenarios (handout or QR code below)

Scenario 1

Questions, for pairs of students

In the scenario, students are asked: What can you conclude from this study?
They should tick the boxes for conclusions they agree with (see linked page)

Discussion

Get each pair to say what they agreed with.
Students often struggle and want to agree with most options

Correct answer: (a) only.

Ask students if they can explain why this is the case?

Notes:

Option b (best performing person used Method B…)

Anecdotal evidence, not based on all data
How psychologists draw conclusions vs media arguments
E.g., “Richard Branson is successful, so copy him” or “My gran smoked and lived to 120”

Option c (didn’t control for other factors) or d (verbal learning style might influence…)

Factors that influence memory but irrelevant here because of randomisation
Often agreed with; common error in student essays
Ask: Can we ever control everything in psychology?

Option e (study list wasn’t unusual/important) or f (just Plymouth students)

These are about generalisation.
All studies happen somewhere, with some materials

Ask:

Where would the study need to be run for you to believe it?
What materials would make the results more important?

Scenario 2

Ask students to consider Scenario 2

This compares recall immediately vs 1 year later.
People “believe” the outcome, so think the conclusion is valid.
It is ‘obvious’ that people won’t recall anything a year later…
Consequently, many students conclude that the study doesn’t need to be run under different conditions. It doesn’t matter that ‘they only tested Plymouth students’ because it seems obvious that the same would happen everywhere.

The key comparison with Scenario 1:

Challenge students directly: (paraphrasing) “For Scenario 1, [some] of you [might have] said we can’t generalise because it only used Plymouth students. But for Scenario 2, you’re saying it doesn’t need testing in other populations. What’s the difference?”

The pedagogical point:

In Scenario 2, there’s a clear mechanism (memory decay over time) that should work similarly across populations
In Scenario 1, it’s less clear whether the specific method would work the same way everywhere.
We should give different answers for the two scenarios, even though both used Plymouth students.

What this reveals:

Students instinctively know that “lacks generalisability” isn’t always a good critique
They apply it inconsistently – based on whether they believe the mechanism
Writing “only tested in Plymouth” or “lacks generalisability” on every study isn’t helpful
Instead, they need to explain why testing in a different population would matter for this specific finding

Discussion points:

“If memory decay works this way in Plymouth, why would it be different in London?”
“What would make you think a finding wouldn’t generalise to other populations?”
This helps students see that generic critiques like “can’t generalise” need to be backed up with a reason why the specific limitation matters

An alternative example (if preferred):

Use a different/clearer mechanism:
e.g., 100g chocolate raises blood sugar more than 100g rice-cakes
Would that need replicating elsewhere?
Would it be important to test students in London as well as Plymouth?

Or make up your own example.

Task 3: Critical analysis practice

Goal: Learn to distinguish generic critiques from specific, thoughtful ones, then practice writing your own.

Topic: Moving from A-level style critiques to university-level critical analysis.

Materials

Research critique examples (handout or QR code below)

Step 1: Spotting weak vs strong critiques (10 minutes)

Show students the QR code or handout

Shows research findings with different critiques
Each example shows a study result followed by limitation statements

Explain the task:

Students will see pairs of critiques. Their job is to:

Decide which critiques are generic (could apply to almost any study) vs specific (identifies particular features that matter)
Decide which are weak vs strong

In pairs (5 minutes):

Look at Examples 1 and 2 only (Loftus & Palmer and Milgram)
For each pair of critiques (A and B): Which is stronger? Why?

Whole group discussion (5 minutes):

Examples 1 & 2 (Loftus & Palmer):

Which critique did you prefer? Why?
Key point: The weak one says “lacks ecological validity because videos not real accidents” - but the study is about how question wording affects memory, not about car accidents specifically. Using videos doesn’t undermine that conclusion.
Strong version: Identifies what might matter (emotional involvement) and why (might interact with suggestibility).

Examples 3 & 4 (Milgram):

Weak: “Can’t generalise, only American men” - stock phrase, doesn’t explain why gender/nationality would matter for obedience
Strong: Identifies specific context features (scientific authority, lab) and why they might create unique demand characteristics

Key points to emphasize:

Generic critiques are swappable - you could say them about any study
Specific critiques identify which aspect of the method might limit which conclusion
Strong critiques explain why the limitation matters

Notice the language in strong critiques: “may differ in important ways”, “it’s unclear whether”, “could influence” - acknowledging uncertainty rather than dismissing findings.

Step 2: Writing your own limitations (10 minutes)

Materials: Study descriptions (handout or QR code below)

Introduce the task:

“Now you’ll practice writing limitations as you would in a discussion section. The handout has 6 study descriptions with methodological detail. Choose one to work with.”

Reiterate: Specific critiques identify which aspect of the method might limit which conclusion

In pairs (7 minutes):

Each pair chooses one study from the handout (or tutor assigns different studies to different pairs)
Discuss: What would you have written at A-level?
Focus on: What aspect of the method might limit which conclusion? Why does it matter?
Write 2-3 sentences identifying specific limitations

Whole group sharing (3 minutes):

2-3 pairs share their study and limitations (1 minute each)
Quick feedback: Did they avoid generic critiques? Did they identify specific features? Did they explain why it matters?

Link to assessment and further practice:

When you write discussion sections, generic critiques don’t earn marks. The handout has more examples if you want extra practice.
There’s also a practice worksheet with additional examples and guidance if you want to develop this skill further.

Example discussion points for each study

Use these if students need prompting during the sharing phase:

Peterson & Peterson - Short Term Memory

Weaker: “Artificial task, lacks ecological validity” (too generic)
Better: “Consonant trigrams have no semantic meaning; in everyday memory we usually remember meaningful material where we can use elaborative strategies. The effect might not apply to meaningful information.”

Godden & Baddeley - Context Dependent Memory

Weaker: “Only used divers, can’t generalise” (too generic)
Better: “Land vs underwater represents an extreme context change; it’s unclear whether the effect would be as strong for more subtle context shifts (e.g., different rooms) which are more typical of everyday memory situations.”

Baddeley - Acoustic and Semantic Similarity

Weaker: “People don’t memorise word lists in real life” (too generic)
Better: “Unrelated word lists isolate similarity effects but may not reflect how similarity affects memory for connected prose or narratives where context provides additional retrieval cues.”

Ainsworth - Strange Situation

Weaker: “Might not generalise to other places” (too generic)
Better: “Brief lab separations may not reflect longer, real-world separations; behaviors might have different meanings in cultures with different independence values.”

Harlow - Contact Comfort

Weaker: “Used monkeys not humans” (too generic)
Better: “Social isolation likely caused severe stress beyond normal rearing conditions; the preference for cloth might reflect distress response rather than normal attachment processes. Also unclear whether binary choice (wire vs cloth) reflects the multiple factors influencing human infant attachment.”

Baby Einstein

Weaker: “Just a correlation” (too generic)
Better: “Parents self-selected; those buying educational media likely differed in unmeasured ways (e.g. reading/talking more to their children, having more free time and disposable income).”

Closing

Ask

What’s one thing from today you’ll remember when writing your practical report discussion?”

Remind

You don’t get credit for generic critiques, only strong specific ones
See the practice worksheet for additional examples to practice with.