Is It Live or Is It Internet? Experimental Estimates of the Effects of Online Instruction on Student Learning
[READ SUBSTANTIALLY BUT NOT COMPLETELY AT SOURCE, the NBER PDF: the abstract, the design, Table 3, Table 4 and the statement of the overall effect were read, and THE FULL REGRESSION DISCUSSION WAS NOT READ. NOT A PEER-REVIEWED ARTICLE AS READ: this is a working paper, later published in the Journal of Labor Economics, and the working paper is the document that was read] Figlio, D. N., Rush, M., & Yin, L. (2010). 'Is It Live or Is It Internet? Experimental Estimates of the Effects of Online Instruction on Student Learning.' NBER Working Paper No. 16089, June 2010. Published as Journal of Labor Economics 31(4), 763-784, 2013. FIGLIO FIRST AUTHOR, Institute for Policy Research, Northwestern. Supported by the National Science Foundation and the United States Department of Education; the university studied is undisclosed in the paper. THE STRONGEST DOING-AGAINST-VIEWING EVIDENCE, AND THE ANSWER EMBARRASSES THE INTUITION. Design: students in a large introductory microeconomics course at a major research university were RANDOMLY ASSIGNED to live lectures or to watching the same lectures online, with the web page, problem sets, teaching-assistant support, office hours and examinations identical across sections; 215 students were in the online-assigned group. Their abstract calls it 'the first experimental evidence on the effects of live versus internet media of instruction' and reports 'modest evidence that live-only instruction dominates internet instruction. These results are particularly strong for Hispanic students, male students, and lower-achieving students.' THE PAPER'S OWN MORE EXACT SENTENCE, verbatim: 'While the overall effect of live instruction relative to internet delivery is very modest and positive (though not statistically distinguishable from zero), these mean effects may mask substantial differences in relative benefits of one medium of instruction over another.' Table 3: live-section average score across three examinations 79.940, standard error 0.850. They note the design can detect effects 'on the order of two points on a 100-point scale'. TABLE 4, HETEROGENEOUS EFFECTS, live minus online in score points with standard errors: Hispanic students 11.276 (3.587), significant at one per cent; male students 3.480 (1.680), significant at five per cent; low achievers 4.054 (1.536), significant at one per cent; White students 1.117 (1.436), Asian students 4.319 (3.590) and female students 1.780 (1.576), all non-significant. Their own gloss: 'These are precisely the students who are more likely to populate the less selective universities and community colleges.' Withdrawal rates were about 6 per cent live against slightly more than 4.5 per cent online. They note that a United States Department of Education meta-analysis of non-experimental studies reached a different conclusion, which their experiment counters. WHAT MAY NOT BE SAID: that live instruction beats online instruction. The overall effect was positive, very modest and not statistically distinguishable from zero, and the subgroup effects are the finding.
[READ SUBSTANTIALLY BUT NOT COMPLETELY AT SOURCE, the NBER PDF: the abstract, the design, Table 3, Table 4 and the statement of the overall effect were read, and THE FULL REGRESSION DISCUSSION WAS NOT READ. NOT A PEER-REVIEWED ARTICLE AS READ: this is a working paper, later published in the Journal of Labor Economics, and the working paper is the document that was read] Figlio, D. N., Rush, M., & Yin, L. (2010). 'Is It Live or Is It Internet? Experimental Estimates of the Effects of Online Instruction on Student Learning.' NBER Working Paper No. 16089, June 2010. Published as Journal of Labor Economics 31(4), 763-784, 2013. FIGLIO FIRST AUTHOR, Institute for Policy Research, Northwestern. Supported by the National Science Foundation and the United States Department of Education; the university studied is undisclosed in the paper. THE STRONGEST DOING-AGAINST-VIEWING EVIDENCE, AND THE ANSWER EMBARRASSES THE INTUITION. Design: students in a large introductory microeconomics course at a major research university were RANDOMLY ASSIGNED to live lectures or to watching the same lectures online, with the web page, problem sets, teaching-assistant support, office hours and examinations identical across sections; 215 students were in the online-assigned group. Their abstract calls it 'the first experimental evidence on the effects of live versus internet media of instruction' and reports 'modest evidence that live-only instruction dominates internet instruction. These results are particularly strong for Hispanic students, male students, and lower-achieving students.' THE PAPER'S OWN MORE EXACT SENTENCE, verbatim: 'While the overall effect of live instruction relative to internet delivery is very modest and positive (though not statistically distinguishable from zero), these mean effects may mask substantial differences in relative benefits of one medium of instruction over another.' Table 3: live-section average score across three examinations 79.940, standard error 0.850. They note the design can detect effects 'on the order of two points on a 100-point scale'. TABLE 4, HETEROGENEOUS EFFECTS, live minus online in score points with standard errors: Hispanic students 11.276 (3.587), significant at one per cent; male students 3.480 (1.680), significant at five per cent; low achievers 4.054 (1.536), significant at one per cent; White students 1.117 (1.436), Asian students 4.319 (3.590) and female students 1.780 (1.576), all non-significant. Their own gloss: 'These are precisely the students who are more likely to populate the less selective universities and community colleges.' Withdrawal rates were about 6 per cent live against slightly more than 4.5 per cent online. They note that a United States Department of Education meta-analysis of non-experimental studies reached a different conclusion, which their experiment counters. WHAT MAY NOT BE SAID: that live instruction beats online instruction. The overall effect was positive, very modest and not statistically distinguishable from zero, and the subgroup effects are the finding.
Written from the rEAD SUBSTANTIALLY BUT NOT COMPLETELY AT SOURCE, the NBER PDF: the abstract, the design, Table 3, Table 4 and the statement of the overall effect were read, and THE FULL REGRESSION DISCUSSION WAS NOT READ. NOT A PEER-REVIEWED ARTICLE AS READ: this is a working paper, later published in the Journal of Labor Economics, and the working paper is the document that was read.
What our sources record
Recorded when this source was checked, figures included, with page references and quotation marks removed.
Figlio, D. N., Rush, M., & Yin, L. (2010). Is It Live or Is It Internet? Experimental Estimates of the Effects of Online Instruction on Student Learning. NBER Working Paper No. 16089