To what extent does AI drafting affect reading scores?
Assignment 2, Intro to Causal Inference
PISA 2025, US students
D and Y
D: weekly AI use to draft texts or summarize
Y: PISA reading score
Data: 4,623 US students, about half with questionnaire data
Weekly drafting users score about 38 points lower on reading assessment (p < .001), holding learning use fixed.
Other causes: who causes what
| Variable | Causes D | Causes Y | In PISA? | |
|---|---|---|---|---|
| Family SES | yes | yes | yes (ESCS) | |
| Prior reading skill | yes | yes | no (proxy: grade level) | |
| Intrinsic motivation | yes | yes | no proxies | |
| Extrinsic motivation | yes | yes | no | |
| Tech propensity | yes | no | no proxies | |
| School AI policy | yes | no | indirect (but pretty much no) | |
| School academic rigor | no | yes | no | |
| Device access | yes | no | yes (ICT resources) |
Family SES: what ESCS measures
| Component | What it captures | |
|---|---|---|
| HISCED | Highest education level of either parent | |
| HISEI | Highest occupational status of either parent | |
| HOMEPOS | Index of household possessions |
Needs and motivation DAG
Ten back-door paths, none blocked
D ← Family → Y
D ← Family → Reading → Y
D ← Family → Reading → Intrinsic → Y
D ← Reading → Y
D ← Reading → Intrinsic → Y
D ← Reading ← Family → Y
D ← Intrinsic → Y
D ← Intrinsic ← Reading → Y
D ← Intrinsic ← Reading ← Family → Y
D ← Extrinsic → Y
What would block them
Only sufficient set: Family, Prior reading, Intrinsic, Extrinsic
Observed-only set: none
Potential proxies: grade level repeat for Prior reading; goal setting and self-regulation for Intrinsic motivation (but could also be a proxy for extrinsic...)
Curiosity and perseverance have no US data.
Takeaways
The effect is not identified from PISA. There are three required variables PISA does not measure.