Hartley nodded. "So we keep the software, but we train Mr. Abel on it too."
"Lena, look," Hartley said, tapping his desk. "I installed it in Ms. Chen’s third-grade class. She’s our best teacher. The other third-grade class, Mr. Abel’s, is using the old curriculum. After three months, I’ll compare their test scores. Simple, right?"
Hartley frowned. "So I should flip a coin? Randomly assign kids to software or no software?" quasi-experimentation a guide to design and analysis pdf
But to be rigorous, she added a and used Huber-White robust standard errors (because monthly scores from the same class aren’t independent — a key point from quasi-experimental guides).
"Exactly," Lena said. "And next time, if you can’t randomize, use a — give half the classes the software in Phase 1, the other half in Phase 2. Compare each against itself over time." Hartley nodded
Result: The +7 points was statistically significant (p < .01) and practically meaningful. Lena presented to Hartley: "The software works, but only by 7 points, not the 15-point jump you saw in the raw comparison. The raw difference was inflated by Ms. Chen’s prior excellence."
Lena sighed. "That’s not simple. That’s a . Ms. Chen is a star teacher. Her kids were already scoring 15% higher before the software. If her class does better afterward, was it the software or just… Ms. Chen?" "I installed it in Ms
Here’s a short, engaging story that captures the essence of (as in the spirit of Cook, Campbell, and Shadish’s work, often summarized in guides like Quasi-Experimentation: A Guide to Design and Analysis ). Title: The Principal’s Predicament Dr. Lena Torres, a research consultant, faced a familiar problem. The school principal, Mr. Hartley, had just spent $50,000 on a new "MindGrow" reading software. He needed to know if it worked.
Lena smiled. "That’s the guide to design and analysis. No randomization? No problem. Just more thinking." Quasi-experimentation isn’t “second-best.” It’s a toolkit for causal inference when experiments are impossible. Master the threats (history, selection, maturation, regression), choose a design (ITS, DID, nonequivalent groups), and analyze with care — robust standard errors and pre-trend checks are your friends.
Hartley laughed. "You quasi-people have a workaround for everything."
"You can’t," Lena said. "Parents would riot if their kid got ‘no software.’ Plus, the software is tied to Ms. Chen’s classroom computers. You have a —real-world, no randomization. But that doesn’t mean it’s hopeless."