Online Tutoring by College Volunteers: Experimental Evidence from a Pilot Program

icpsr_169121

Matthew A. Kraft, John A. List, Jeffrey A. Livingston, Sally Sadoff (2022). EdWorkingPaper No. 22-568, Annenberg Institute at Brown University

DOI: https://doi.org/10.26300/b1ch-0g29

field	value
venue	EdWorkingPaper No. 22-568, Annenberg Institute at Brown University
discipline	education
paper	https://doi.org/10.26300/b1ch-0g29
replication data	openICPSR 169121
data license	CC BY 4.0 — “This work is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License.”
regression datasets (in this corpus)	3
graduated / eligible / exported	3 / 8 / 15 (policy `claims_manual_regression_cap`)
routed / gap / residue (at source)	16 / 62 / 0

Claims & tasks

Every claim the paper makes in its main body about a parameter, joined to the benchmark task (dataset · coefficient) that captures it. The effect, s.e. and p are the captured regression's own reported values (the paper's Stata figures, carrying the paper's SE method); the row shows the paper's verbatim quote (page · exhibit), so you can confirm the regression matches what the paper says. The prominence column is the within-study inclusion signal (⭐ headline → primary → secondary). The sensitivity column is the privacy level of the most-sensitive variable in the regression (🔴 high / 🟠 medium / ⚪ low).

#	prominence	sensitivity	claim	effect	s.e.	p	task	instance `(dataset · coef)`
C1	primary	🔴 high	Pooled ITT and TOT estimates are 0.053σ and 0.66σ, respectively. p. 5 · Table 1, col 1 (Pooled tests)	+0.053	0.043	0.214	Linear Regression	`reg_1 · treatment` n=2092 · d=23
C2	secondary	🔴 high	Our pooled ITT estimate for waves 2 and 3 is correspondingly smaller, 0.021σ. p. 6 · Table 1	+0.021	0.059	0.727	Linear Regression	`reg_11 · treatment` n=1080 · d=20
C3	secondary	🔴 high	Estimates for the pooled ITT effects on achievement are 0.087σ for wave 1 where students attended an average of 4.0 hours of tutoring. p. 6 · Table 1	+0.087	0.062	0.164	Linear Regression	`reg_6 · treatment` n=1012 · d=17

Regression datasets

The 3 regression dataset(s) graduated into the benchmark corpus from this study's replication package — each backs one or more claims above. Expand each for its features, response, (sound, data-independent) public bounds, and reproduction grade.

icpsr_169121_reg_1
icpsr_169121_reg_11
icpsr_169121_reg_6

icpsr_169121_reg_1 — testscore_s ~ treatment + math_s + math_s_2 + reading_s + reading_s_2 + math_s_imputed + …

Sensitivity proposal: 🔴 high maximum across columns (advisory; reviewed manually). Reproduction grade: ✅ match (benchmark-eligible).

Features

name	description	type	sensitivity	bound lo	bound hi	estimate	s.e.
`_cons`	intercept	continuous	—	1	1	+0.1763	0.122
`treatment`	Random assignment to tutoring treatment arm	continuous	⚪ low	0.0	1.0	+0.05334	0.0429
`math_s`	Baseline standardized math test score	continuous	🟠 medium	-6.0	6.0	+0.4099	0.0402
`math_s_2`	Squared baseline standardized math score	continuous	🟠 medium	0.0	36.0	+0.02434	0.0236
`reading_s`	Baseline standardized reading test score	continuous	🟠 medium	-6.0	6.0	+0.4359	0.0373
`reading_s_2`	Squared baseline standardized reading score	continuous	🟠 medium	0.0	36.0	-0.01317	0.0223
`math_s_imputed`	Indicator baseline math score was imputed	continuous	⚪ low	0.0	1.0	-0.1185	0.111
`reading_s_imputed`	Indicator baseline reading score was imputed	continuous	⚪ low	0.0	1.0	-0.459	0.135
`female`	Student gender indicator (female)	continuous	🟠 medium	0.0	1.0	-0.02296	0.0443
`Black`	Student race indicator: Black	continuous	🔴 high	0.0	1.0	-0.1175	0.127
`Hispanic`	Student ethnicity indicator: Hispanic	continuous	🔴 high	0.0	1.0	-0.03817	0.119
`ell`	English language learner status indicator	continuous	🟠 medium	0.0	1.0	-0.0498	0.0576
`checc_control`	Control-group site/partner indicator	continuous	⚪ low	0.0	1.0	+0.02675	0.0559
`checc_KP`	Site/partner indicator, CHECC KP program	continuous	⚪ low	0.0	1.0	-0.02722	0.0892
`checc_PA`	Site/partner indicator, CHECC PA program	continuous	⚪ low	0.0	1.0	+0.1518	0.0793
`checc_PK`	Site/partner indicator, CHECC PK program	continuous	⚪ low	0.0	1.0	+0.0678	0.0759
`grade6_wave2`	Grade 6 by wave 2 interaction dummy	continuous	⚪ low	0.0	1.0	-0.05845	0.0777
`grade6_wave3`	Grade 6 by wave 3 interaction dummy	continuous	⚪ low	0.0	1.0	+0.1255	0.121
`grade7_wave1`	Grade 7 by wave 1 interaction dummy	continuous	⚪ low	0.0	1.0	-0.01677	0.0783
`grade7_wave2`	Grade 7 by wave 2 interaction dummy	continuous	⚪ low	0.0	1.0	-0.2077	0.0785
`grade7_wave3`	Grade 7 by wave 3 interaction dummy	continuous	⚪ low	0.0	1.0	-0.2082	0.103
`grade8_wave1`	Grade 8 by wave 1 interaction dummy	continuous	⚪ low	0.0	1.0	-0.04176	0.0717
`grade8_wave2`	Grade 8 by wave 2 interaction dummy	continuous	⚪ low	0.0	1.0	-0.0593	0.0732
`grade8_wave3`	Grade 8 by wave 3 interaction dummy	continuous	⚪ low	0.0	1.0	-0.2267	0.127

Response

name	description	type	sensitivity	bound lo	bound hi	estimate	s.e.
`testscore_s`	Pooled standardized student achievement test score	continuous	🟠 medium	-6.0	6.0	—	—

Public bounds (data-independent) sourced from: binary indicator; binary indicator; treatment/arm variable; binary interaction indicator; square of standardized score; standardized (_s) score.

n = 2,092 samples.

icpsr_169121_reg_11 — testscore_s ~ treatment + math_s + math_s_2 + reading_s + reading_s_2 + math_s_imputed + …

Sensitivity proposal: 🔴 high maximum across columns (advisory; reviewed manually). Reproduction grade: ✅ match (benchmark-eligible).

Features

name	description	type	sensitivity	bound lo	bound hi	estimate	s.e.
`_cons`	intercept	continuous	—	1	1	+0.1253	0.189
`treatment`	Random assignment to tutoring treatment arm	continuous	⚪ low	0.0	1.0	+0.02069	0.0591
`math_s`	Baseline standardized math test score	continuous	🟠 medium	-6.0	6.0	+0.3923	0.0573
`math_s_2`	Squared baseline standardized math score	continuous	🟠 medium	0.0	36.0	+0.04018	0.0303
`reading_s`	Baseline standardized reading test score	continuous	🟠 medium	-6.0	6.0	+0.4663	0.0559
`reading_s_2`	Squared baseline standardized reading score	continuous	🟠 medium	0.0	36.0	-0.02879	0.035
`math_s_imputed`	Indicator baseline math score was imputed	continuous	⚪ low	0.0	1.0	-0.03872	0.117
`reading_s_imputed`	Indicator baseline reading score was imputed	continuous	⚪ low	0.0	1.0	-0.5862	0.172
`female`	Student gender indicator (female)	continuous	🟠 medium	0.0	1.0	+0.003595	0.0638
`Black`	Student race indicator: Black	continuous	🔴 high	0.0	1.0	-0.1468	0.193
`Hispanic`	Student ethnicity indicator: Hispanic	continuous	🔴 high	0.0	1.0	-0.05958	0.186
`ell`	English language learner status indicator	continuous	🟠 medium	0.0	1.0	+0.02946	0.0776
`checc_control`	Control-group site/partner indicator	continuous	⚪ low	0.0	1.0	+0.001344	0.088
`checc_KP`	Site/partner indicator, CHECC KP program	continuous	⚪ low	0.0	1.0	+0.07308	0.116
`checc_PA`	Site/partner indicator, CHECC PA program	continuous	⚪ low	0.0	1.0	+0.1039	0.112
`checc_PK`	Site/partner indicator, CHECC PK program	continuous	⚪ low	0.0	1.0	-0.0245	0.0916
`grade6_wave3`	Grade 6 by wave 3 interaction dummy	continuous	⚪ low	0.0	1.0	+0.2079	0.128
`grade7_wave2`	Grade 7 by wave 2 interaction dummy	continuous	⚪ low	0.0	1.0	-0.1481	0.0839
`grade7_wave3`	Grade 7 by wave 3 interaction dummy	continuous	⚪ low	0.0	1.0	-0.1487	0.114
`grade8_wave2`	Grade 8 by wave 2 interaction dummy	continuous	⚪ low	0.0	1.0	+0.02352	0.0813
`grade8_wave3`	Grade 8 by wave 3 interaction dummy	continuous	⚪ low	0.0	1.0	-0.1505	0.136

Response

name	description	type	sensitivity	bound lo	bound hi	estimate	s.e.
`testscore_s`	Pooled standardized student achievement test score	continuous	🟠 medium	-6.0	6.0	—	—

Public bounds (data-independent) sourced from: binary indicator; binary indicator; treatment/arm variable; binary interaction indicator; square of standardized score; standardized (_s) score.

n = 1,080 samples.

icpsr_169121_reg_6 — testscore_s ~ treatment + math_s + math_s_2 + reading_s + reading_s_2 + math_s_imputed + …

Sensitivity proposal: 🔴 high maximum across columns (advisory; reviewed manually). Reproduction grade: ✅ match (benchmark-eligible).

Features

name	description	type	sensitivity	bound lo	bound hi	estimate	s.e.
`_cons`	intercept	continuous	—	1	1	+0.2119	0.12
`treatment`	Random assignment to tutoring treatment arm	continuous	⚪ low	0.0	1.0	+0.08704	0.0623
`math_s`	Baseline standardized math test score	continuous	🟠 medium	-6.0	6.0	+0.4134	0.0563
`math_s_2`	Squared baseline standardized math score	continuous	🟠 medium	0.0	36.0	+0.001056	0.0373
`reading_s`	Baseline standardized reading test score	continuous	🟠 medium	-6.0	6.0	+0.414	0.0505
`reading_s_2`	Squared baseline standardized reading score	continuous	🟠 medium	0.0	36.0	-5.8e-05	0.0297
`math_s_imputed`	Indicator baseline math score was imputed	continuous	⚪ low	0.0	1.0	-0.1844	0.211
`reading_s_imputed`	Indicator baseline reading score was imputed	continuous	⚪ low	0.0	1.0	-0.3581	0.216
`female`	Student gender indicator (female)	continuous	🟠 medium	0.0	1.0	-0.05118	0.0613
`Black`	Student race indicator: Black	continuous	🔴 high	0.0	1.0	-0.1404	0.122
`Hispanic`	Student ethnicity indicator: Hispanic	continuous	🔴 high	0.0	1.0	-0.06727	0.114
`ell`	English language learner status indicator	continuous	🟠 medium	0.0	1.0	-0.1177	0.0849
`checc_control`	Control-group site/partner indicator	continuous	⚪ low	0.0	1.0	+0.05445	0.0744
`checc_KP`	Site/partner indicator, CHECC KP program	continuous	⚪ low	0.0	1.0	-0.07293	0.132
`checc_PA`	Site/partner indicator, CHECC PA program	continuous	⚪ low	0.0	1.0	+0.1947	0.111
`checc_PK`	Site/partner indicator, CHECC PK program	continuous	⚪ low	0.0	1.0	+0.1429	0.115
`grade7`	Student is enrolled in grade 7	continuous	🟠 medium	0.0	1.0	-0.0195	0.077
`grade8`	Student is enrolled in grade 8	continuous	🟠 medium	0.0	1.0	-0.05116	0.0748

Response

name	description	type	sensitivity	bound lo	bound hi	estimate	s.e.
`testscore_s`	Pooled standardized student achievement test score	continuous	🟠 medium	-6.0	6.0	—	—

Public bounds (data-independent) sourced from: binary indicator; binary indicator; treatment/arm variable; square of standardized score; standardized (_s) score.

n = 1,012 samples.