Does Personal Relevance Matter?

Testing personalized environmental stories

The Big Question

Can we change how participants think about environmental action by making stories personally relevant?

What We Mean by "Personal"

🏠 Hometown-specific content

💧 Top environmental concern (their choice)

vs.

📰 Generic environmental story

The Experiment

N = 296 participants

Randomly assigned to:

✅ Treatment: Story tailored by Anthropic's API on the spot to their hometown + concern

⚪ Control: Less relevant story

First: Did the Manipulation Work?

(Did people actually perceive the difference?)

✓ Yes, It Worked

Relevance to their concern:

t = -7.63, p < .001

Relevance to their place:

t = -4.44, p < .001

Treatment group found stories significantly more relevant

We Measured...

📚 Learning: Quiz on story content

🌍 Awareness: Environmental concern frequency

😐 Affect: Mood and climate anxiety

✅ Behavior: Willingness to take action

So... Did They Learn More?

The results get interesting

Learning: What Worked

One question showed strong effects:

(broken into true/false items, all significant p < .01)

Treatment group scored higher on this content

Learning: The Bigger Picture

Across all learning questions?

Statistical significance ✗

They learned something specific, but not everything

One measure went the opposite direction

Effect Sizes: Reality Check

learn2_avg:

Control = 1.35

Treatment = 1.45

Difference = 0.10 on a scale

Statistically significant ✓

Practically meaningful ?

What We Have

✅ Manipulation worked perfectly

✅ Some significant learning effects

⚠️ But: Small effect sizes

⚠️ And: Mixed/inconsistent patterns

Help!

This is proof of concept...

But how do we make it stronger?

Methodological Questions

Measurement:

Are we capturing learning appropriately?

Better scales or items?

Intervention:

Was one story too light-touch?

Wrong timing for assessment?

Sample:

Missing important moderators?

Your Expertise

What would you change about:

🔬 The design?

📏 The measures?

⚡ The intervention?

Let's Talk

We're here to learn from you!

Questions? Suggestions? Critiques?