Research at Edmentum: Teaming Up with Marzano Research to Understand How Study Island Drives Student Growth

If you are a fan of Dr. Robert J. Marzano, you probably have some idea of why I was so excited to write this blog post. Marzano is a leading researcher in education. He is the author of more than 40 books and over 300 articles, and he is currently the chief academic officer of Marzano Research. After all these years, my favorite of his publications remains Classroom Instruction that Works (2001), now available in a more recent edition. I had the pleasure of instructing a new teacher graduate course with this as the required text. It was so much fun working with these enthusiastic first-year educators to learn how to apply Marzano’s meta-analyses of best practices. If Marzano had an official “tour,” I’d be the person constantly listening to the radio, hoping to win VIP passes and front-row tickets! So, you can imagine how eager our Research Team here at Edmentum was to work on our Study Island research design and scientific study with Marzano Research.

While education has always been research driven, the Every Student Succeeds Act (ESSA) and the ever-changing policy landscape have placed an increasing emphasis on evidence-based practices. However, there is often confusion as to what exactly qualifies as evidence-based practices. Our Research Team here at Edmentum was honored to have Marzano Research peer review a recent research project that we conducted on our program for classroom practice and assessment, Study Island. In our original research, we used What Works Clearinghouse™ (WWC) standards to measure the impact of regular Study Island use on student achievement. Study Island earned a very good WWC ranking—“moderate evidence”—that meets requirements of virtually all state education agencies (SEAs) and local education agencies (LEAs) in the United States. WWC standards apply the most rigorous of research review, based on scientific evidence, and the organization exists to help educators and policymakers make decisions that are based on high-quality research supported by scientific evidence. The organization has developed the highest standards in peer review and is part of the Institute of Education Sciences (IES), which is an agency within the U.S. Department of Education dedicated to performing statistics research and evaluation. The WWC seeks to answer the question: “What works in education?” Results of our study indicate that Study Island works really well, and our collaboration with Marzano Research for peer review confirms that there is scientific evidence to support this.

Our Research Design: Ensuring That Study Island Efficacy Meets a High Bar of Evidence

Conducting a peer review to prove scientific evidence is a process, not an event. Such processes take time and collaboration between the peers—typically post-doctorate or graduate school researchers who are experts in their field. In working with Marzano Research, our goal was to confirm that our internal research on Study Island constituted a properly executed quasi-experimental design that was rigorous, as defined by WWC standards. Our quantitative study of the effectiveness of our Study Island online learning platform on student growth suggests that Study Island is effective for use in elementary school classrooms across the country. The Marzano Research peer review verifies that our study was rigorously performed and, therefore, meets the criteria for what the U.S. Department of Education (2016) refers to as “moderate evidence” for effective classroom interventions. Earning the “moderate evidence” designation in such a peer review is significant, as many studies do not stand up to WWC standards. Thus, meeting such standards, the research study may be used to support the specific classroom intervention of using the Study Island standards mastery product in elementary schools.

So, what exactly did our own research uncover—as well as the Marzano Research peer review—to demonstrate that Study Island meets ESSA and WWC evidence standards? Here’s an overview of our findings.

Study Island Usage Leads to Academic Growth and Achievement Gains

Edmentum’s research study found that students experienced significant growth or achievement gains with Study Island. The intervention was defined as the availability of Study Island for a period of 12 weeks by elementary school students in the fall and winter of 2016–17. The outcome variable was student growth in achievement scores using Edmentum’s highly reliable and valid Exact Path adaptive diagnostic growth scale.

The following tables show the differences between treatment and control groups across grade for each subject with the scale score range on the left. The treatment effect evidences student achievement gains and is consistent throughout.

Only 30 Minutes a Week on Study Island Leads to Student Growth

Beyond evaluating the effectiveness of Study Island to drive growth, our Edmentum Research Team was also interested in finding out exactly how much time a learner needs to use Study Island to experience those achievement gains. Essentially, we wanted to determine if there is a relationship between the amount of time students spend engaged in the Study Island platform and the academic growth they attain. Our results demonstrate what recent research from Alan Cheung and Robert Slavin has already confirmed: that modern educational technology must be used faithfully to realize its potential. The authors noted that modern, computer-assisted instructional programs should be used for about 30 minutes per week to realize benefits. Yes, you read that correctly—just 30 minutes a week.

Typical growth for learners whose average use was approximately in line with Cheung and Slavin’s recommendation was a full 50 points on Edmentum’s Exact Path developmental growth scale. This finding is considered robust. The following tables show the five equal groups (quintiles) into which the population was divided according to the distribution. The scaled score growth is indicated with the Study Island usage time (by minutes) per week. As you can see, students who were engaged in Study Island for 20 to 30 minutes each week had the highest growth.

As educators, studies such as this are a good reminder to stay grounded in routines, or what Cheung and Slavin call “faithful usage.” We’ve all had our moments when we’ve had to compromise the routine—especially during those particularly chaotic times of the school year just before winter, spring, and summer breaks. However, here we have scientific evidence, backed up by Marzano Research’s peer review, that just 30 minutes a week leads to growth using Study Island’s mastery model. It gives good argument for committing to set that time aside for Study Island usage. Practice with Study Island doesn’t have to dominate your instructional time, but regularly incorporating it in small, manageable chunks can make a big difference for your students.

Interested in learning more about the research behind Study Island? Read our quasi-experimental report summary indicating that Study Island meets ESSA requirements of an “evidence-based” intervention, and watch for additional research from our own Research Team and Marzano Research in the months to come!