“Evidence Based” “Proven” “Scientific” “Based on Research”
With even a casual interest, it should come as no surprise that these phrases have accompanied almost every release of an educational product in the past decade in either old or new technology. From across the spectrum of new curriculum textbooks, Ed Tech software programs or gameification apps there is the accompanying assertion of some sort of underlying proof.
This puffery has been omnipresent since 2001 when the phrase “scientifically based research” was mentioned 111 times in the No Child Left Behind Act. The law compelled educators to use “teaching practices that have been proven to work.” And in order to receive federal funds, eligible school districts were mandated to submit a proposal to their state outlining how they planed to teach their students to read using “research based practices.”
Thus, it was ”illegal” to use a product that was not “research based.” Therefore, it behooved everyone to make the assertion of scientific discipline, if they wanted to sell their product to schools in the US.
That said, it is also important to know that not all research is created equal. In fact, it wasn’t until 2013, after several peer reviewed studies were unable to show any statistically significant outcomes from products making such claims, that the US Department of Education issued a draft codification of the level of rigor we should expect under a claim of “evidence based.” The SEVEN randomized control trials proving the effectiveness and promise of literacy outcomes delivered by Learning Ovations, Inc, are seen as the “gold standard” of research and have been validated in 23 peer reviewed journal articles.
Collecting Data is Not Enough
To begin with, let’s agree “simply collecting data is not in and of itself scientific.” Sure, collecting all sorts of data can be extraordinarily helpful to make classroom decisions and differentiating instruction to meet individual students’ needs. And there’s nothing wrong with the informal gathering of information, especially if it is coupled with trial teaching and corrective feedback. The difference between these informal, often intuitive activities and research that informs practice is that research is about testing out ideas, drawing conclusions, seeing whether these conclusions hold true at different points in time, and knowing whether they hold true in different situations.
3 Levels of Research and What Constitutes Rigor.
As you are assessing the various claims of providers of education products and technologies in the market, Superintendents, Principals, Teachers, Parents and Boards of Education need to be more discerning as to the claims being made. The following chart will help you categorize the type of research being claimed and will enable you to compare the marketing claim with what is permissible/factual given the level of research rigor. At present, Learning Ovations, Inc, provides the ONLY elementary education system that has applied the highest level of rigor in seven separate randomized control research trials and can therefore claim to improve ALL children’s literacy outcomes in existing classroom settings!
Our Research: A2i Literature Review
Selected References (all references available upon request):
Al Otaiba, S., Carol M. Connor, Jessica S. Folsom, L. Greulich, J. Meadows and Z. Li (2011). “Assessment Data–Informed Guidance to Individualize Kindergarten Reading Instruction: Findings from a Cluster-Randomized Control Field Trial.” The Elementary School Journal 111(4): 535-560.
Bos, C., N. Mather, R. F. Narr and N. Babur (1999). “Interactive, collaborative professional development in early literacy instruction: Supporting the balancing act.” Learning Disabilities Research and Practice 14(4): 227-238.
Chall, J. S. (1967). Learning to read: The great debate. New York, McGraw-Hill Book Co.
Connor, C. M. (2011). Child by Instruction interactions: Language and literacy connections. Handbook on early literacy. S. B. Neuman and D. K. Dickinson. New York, Guilford: 256-275.
Connor, C. M. (2012). Intervening to support reading comprehension development with diverse learners. Unraveling the Behavioral, Neurobiological and Genetic Components of Reading Comprehension: The Dyslexia Foundation and NICHD. B. Miller and L. E. Cutting. Estonia: in press.
Connor, C. M., B. Fishman, E. Crowe, P. Underwood, C. Schatschneider and F. J. Morrison (in press). Third grade teachers’ use of Assessment to Instruction (A2i) software and students’ reading comprehension gains. In press, Technology for literacy achievements for children at risk. O. Korat and A. Shamir. NY, Springer.
Connor, C. M., S. R. Goldman and B. Fishman (in press). Reading and writing technology. Handbook of Research on Educational Communications and Technology. M. Spector, D. Merrill and M. J. Bishop, Association for Educational Communications and Technology.
Connor, C. M. and F. J. Morrison (2012). Knowledge Acquisition in the Classroom: Literacy and Content Area Knowledge Knowledge Development in Early Childhood: How Young Children Build Knowledge and Why It Matters. A. M. Pinkham, T. Kaefer and S. B. Neuman. New York, Guilford Press in press: 220-241.
Connor, C. M., F. J. Morrison, B. Fishman, E. C. Crowe, S. Al Otaiba and C. Schatschneider (in press). “A Longitudinal Cluster-Randomized Control Study on the Accumulating Effects of Individualized Literacy Instruction on Students’ Reading from 1st through 3rd Grade.” Psychological Science.
Connor, C. M., F. J. Morrison, B. Fishman, S. Giuliani, M. Luck, P. Underwood, A. Bayraktar, E. C. Crowe and C. Schatschneider (2011). “Classroom instruction, child X instruction interactions and the impact of differentiating student instruction on third graders’ reading comprehension.” Reading Research Quarterly 46(3): 189-221.
Connor, C. M., F. J. Morrison, B. Fishman, C. C. Ponitz, S. Glasney, P. Underwood, S. Piasta, E. Crowe and C. Schatschneider (2009). “The ISI classroom observation system: Examining the literacy instruction provided to individual students.” Educational Researcher 38(2): 85-99.
Connor, C. M., F. J. Morrison, B. Fishman and C. Schatschneider (in press). Assessment and instruction connections: The implications of child X instruction Interactions effects on student learning. Assessing Reading in the 21st Century: Aligning and Applying Advances in the Reading and Measurement Sciences. J. Sabatini and E. R. Albro. Lanham, MD, R& L Education.
Connor, C. M., F. J. Morrison, B. J. Fishman, C. Schatschneider and P. Underwood (2007). “THE EARLY YEARS: Algorithm-guided individualized reading instruction.” Science 315(5811): 464-465.
Connor, C. M., F. J. Morrison and E. L. Katch (2004). “Beyond the reading wars: The effect of classroom instruction by child interactions on early reading.” Scientific Studies of Reading 8(4): 305-336.
Connor, C. M., F. J. Morrison, C. Schatschneider, J. Toste, E. G. Lundblom, E. Crowe and B. Fishman (2011). “Effective classroom instruction: Implications of child characteristic by instruction interactions on first graders’ word reading achievement.” Journal for Research on Educational Effectiveness 4(3): 173-207.
Connor, C. M., S. B. Piasta, B. Fishman, S. Glasney, C. Schatschneider, E. Crowe, P. Underwood and F. J. Morrison (2009). “Individualizing student instruction precisely: Effects of child by instruction interactions on first graders’ literacy development.” Child Development 80(1): 77-100.
Connor, J. A. and S. Kadel-Taras (2002). Community visions, community solutions: A systems approach to problem-solving. St. Paul, MN, Wilder.