Personal Relevance

Initial results

Manipulation Check

✅ Manipulation Check Results

Variable Control Mean Treatment Mean p-value
manip_place_17 43.27 72.56 0.052
manip_concern_17 54.64 84.33 0.032

🔹 Treatment participants perceived stories as significantly more relevant

📊 Engagement Scores by Group

Variable Description Control Mean (SD) Treatment Mean (SD) p-value
engage1 How engaging was story 3.73 (±1.01) 3.22 (±0.44) 0.18
engage2 Interest in story 3.73 (±1.35) 3.67 (±0.50) 0.90
engage3 Resonance with Alex 3.09 (±1.45) 3.44 (±1.01) 0.54

✅ Trend: Control group reported higher engagement
💭 ...but interest is a wash at the moment. What is the difference between engaging and interesting to the participant?

🤔 But Why Higher Engagement in Control?

I examined:

  • Whether high relevance in control group could explain high engagement
  • Whether engagement was driven more by perceived relevance than group

Correlation results:

Pair r (correlation)
engage1 ↔ manip_concern 0.03
engage1 ↔ manip_place 0.05

❌ No relationship between perceived relevance and engagement

🤔 Is the question worded in the wrong way?

🔍 Regression: Predicting Engagement

Predictor Coef. p-value Interpretation
treat –0.73 0.11 Treatment group slightly lower on engagement
manip_concern_17 +0.004 0.63 No effect
manip_place_17 +0.004 0.63 No effect

🔹 None of the predictors were significant
🔹 Engagement appears unrelated to relevance or treatment

Personal Relevance

Initial results

None of the emotions were statistically significant; considering them a wash for now.

✅ Summary Statistics by Group

Variable Concept Control Mean (SD) Treatment Mean (SD) p-value
assoc Felt association with story 3.09 (±1.22) 3.22 (±0.83) 0.79
identity Story connected to self-identity 3.27 (±1.35) 3.00 (±1.22) 0.65
use Intention to apply/share story learnings 3.64 (±1.29) 2.89 (±0.93) 0.16 ⬇️

🔁 Correlations

Variable Pair Correlation (r)
assoc ↔ identity 0.62
identity ↔ use 0.68
use ↔ engage1 0.81 ✅
assoc ↔ engage1 0.40
identity ↔ engage1 0.48

🧠 Engagement is strongly associated with intended use

Group Differences on Nature Connection Measures (Control – Treatment)

Nature Connection Measure Mean (Control) Mean (Treatment) Diff p-value Interpretation
How often do you feel captivated by nature? 3.91 3.78 +0.13 0.805 No significant difference
How important is your relationship with nature to your identity? 3.73 3.11 +0.62 0.325 Non-significant trend favoring Control
How comfortable or uncomfortable are you in nature? 5.91 5.44 +0.46 0.481 No significant difference
How closely does nature align with your identity? 4.64 4.44 +0.19 0.822 No significant difference

🧠 Knowledge Score Results (Pilot)

Group Mean Score (out of 5) SD n
Control 3.55 1.57 11
Treatment 3.67 1.22 9
Total 3.60 1.39 20
  • Mean difference: -0.12 (control slightly lower)
  • t(18) = -0.19, p = 0.85 — no statistically significant difference
  • 95% Confidence Interval: [-1.47, 1.23]

🧠 Item-Level Knowledge Accuracy (Pilot, n=20)

Item Topic Control Mean Treatment Mean Mean Diff p-value Notable?
learn1 Hottest decade (125k years) 0.73 0.67 +0.06 0.78 ❌ No difference
learn2 Species shifting north/higher altitudes 0.91 0.78 +0.13 0.44 ❌ No difference
learn3 57 companies = 70% emissions 0.73 0.67 +0.06 0.78 ❌ No difference
learn4 14.5M climate deaths (WEF) 0.73 0.78 –0.05 0.81 ❌ No difference
learn5 $12.5T in economic losses (WEF) 0.45 0.78 –0.32 0.16 ⚠️ Trend (treat > ctrl)

🔍 Key Takeaways

  • Most items show no clear difference between groups — performance was high across the board.
  • learn5 (economic losses) shows a potentially meaningful effect:
  • Treatment group was ~32 percentage points more accurate
  • p = 0.16 — not significant, but most promising candidate
  • Suggests that personalization might improve recall of high-level economic facts

🌿 Future Environmental Behavior Intentions: T-Test Results

Variable Behavior Intention Control Mean (SD) Treatment Mean (SD) Mean Diff p-value Notable?
fbx1 Talk to others about environmental issues 3.18 (±1.54) 3.44 (±1.51) –0.26 0.71 ❌ No difference
fbx2 Consider environment in purchases 3.82 (±1.47) 3.33 (±1.41) +0.48 0.47 ❌ No difference
fbx4 Persuade friends to be more eco-friendly 3.64 (±1.86) 3.78 (±1.56) –0.14 0.86 ❌ No difference

🔍 Interpretation

  • All behavior intentions hover around the midpoint of the 1–5 scale.
  • None of the items show statistically significant differences (all p > 0.45).
  • The largest (though still non-significant) difference is in fbx2 (purchase intention), with control slightly higher than treatment.
  • Given the high standard deviations and small sample (n=20), no strong conclusions can be drawn — but patterns can inform future design.

♻️ Personal Environmental Influence (PEI): T-Test Results

Variable Question Control Mean (SD) Treatment Mean (SD) Mean Diff p-value Notable?
pei1 How much of a difference can your personal actions have on the climate? 3.00 (±1.73) 3.22 (±1.72) –0.22 0.78 ❌ No difference
pei2 How much power do you have to help protect the environment? 3.27 (±1.90) 3.00 (±1.66) +0.27 0.74 ❌ No difference
pei3 How often do you think about how your actions affect the environment? 3.91 (±1.81) 3.33 (±1.66) +0.58 0.47 ❌ No difference
pei4 How optimistic are you about helping friends act sustainably? 3.36 (±1.86) 2.89 (±1.76) +0.47 0.57 ❌ No difference

🧠 Interpretation

  • PEI ratings are moderate (means ≈ 3) across both groups, indicating neither high nor low confidence in influence.
  • No statistically significant differences between treatment and control on any item (all p > .47).
  • pei3 and pei4 show numerical gaps in favor of control, but not enough to be meaningful given sample size and SDs.

Summary of Key Variables

Variable Concept Control Mean (SD) Treatment Mean (SD) p-value Notable?
yale1 Worry about global warming 3.18 (±1.17) 3.44 (±0.53) 0.54 ❌ No difference
yale2 Harm to self 2.73 (±1.10) 2.78 (±0.83) 0.91 ❌ No difference
yale3_dist_temporal When harm will occur in U.S. (↑ = sooner) 3.82 (±1.94) 5.11 (±1.27) 0.10 ⚠️ Treatment sees harm sooner
yale4 Harm to future generations 3.45 (±0.93) 3.67 (±0.71) 0.58 ❌ No difference
yale7 Harm to plants/animals 3.45 (±0.93) 3.67 (±0.71) 0.58 ❌ No difference

Climate Concern Index

Composite score (mean of all 5 items):

Group Mean (SD)
Control 3.33 (±1.07)
Treatment 3.73 (±0.62)
p = 0.33 → Not significant

Interpretation

  • Participants show moderate to high concern about climate change across both groups.
  • Only yale3_dist_temporal approaches significance: treatment may delay perceived harm.
  • Climate Concern Index shows a numerical trend favoring treatment, but not statistically significant.

📊 Summary Statistics by Group

Variable Concept Control Mean (SD) Treatment Mean (SD) p-value Notable?
motiv Motivation to act locally 4.00 (±1.61) 3.33 (±1.66) 0.38 ❌ No difference
plearn Perceived learning from story 2.91 (±1.30) 2.89 (±0.93) 0.97 ❌ No difference
action_count Number of local action ideas generated 4.36 (±1.29) 5.00 (±0.00) 0.16 ⚠️ Trend: Treatment wrote more

📊 Summary of Engagement & Distance Variables

Variable Concept Control Mean (SD) Treatment Mean (SD) p-value Notable?
engage1 Story engagement 3.73 (±1.01) 3.22 (±0.44) 0.18 ⚠️ Trend favoring control
engage2 Story interest 3.73 (±1.35) 3.67 (±0.50) 0.90 ❌ No difference
engage3 Resonance with Alex 3.09 (±1.45) 3.44 (±1.01) 0.54 ❌ No difference
dist_areatime Perceived harm to local area (5 yrs) 3.18 (±1.25) 3.33 (±0.71) 0.75 ❌ No difference
dist_social Close others affected 2.55 (±1.69) 2.67 (±0.71) 0.84 ❌ No difference
dist_spatial Already affected locally 3.00 (±1.34) 3.33 (±0.71) 0.51 ❌ No difference
dist_uncertain Likelihood of personal climate impact 2.91 (±1.51) 2.44 (±1.24) 0.47 ❌ No difference
yale3_dist_temporal When harm will affect U.S. 3.82 (±1.94) 5.11 (±1.27) 0.10 ⚠️ Treatment sees harm sooner

🧮 Composite Indices

Index Description Control Treatment p-value Notable?
engage_index Avg of engage1–3 3.52 3.44 0.86 ❌ No difference
dist_index Avg of areatime, spatial, uncertain 3.03 3.04 0.99 ❌ No difference

✅ Notable Findings

  • Manipulation check worked: Treatment group rated stories as significantly more relevant (📍 +30 point difference)
  • Knowledge performance was slightly higher in treatment (not significant), with strong overall accuracy
  • Action step completion was perfect in treatment group (5/5 consistently), with more variability in control
  • Climate concern (yale3_dist_temporal) trended much sooner for treatment

🤔 Curious Results

  • Engagement (engage1) was higher in control, despite receiving non-personalized stories
  • Engagement did not correlate with perceived relevance (manip_*)
  • Regression confirmed no predictors (group or relevance) significantly explained engagement
  • Identity, motivation, and proximity beliefs showed no major group differences

💡 Takeaway

Personal relevance clearly shaped perceptions, but not always in the expected emotional or behavioral ways — suggesting more complex dynamics between personalization, engagement, and action.