[Research] Using Study Island to Support State Assessment Achievement

Student scores and performance-level classifications on state tests are an important part of school accountability under federal and state governments. When schools invest in programs like Edmentum’s Study Island, they want to know that the programs work. During 2018, the Edmentum Research and Design team completed three studies set in three different districts (Reading, Allentown, and an Arizona district) that examined how using Study Island affects students’ performance on state tests. Here’s a summary of the results.

Research Questions

All three studies examined similar research questions:

1. Were there significant differences in the state test scores between students who used Study Island and those who did not?
2. Was there a significant relationship between state test proficiency level categorization and Study Island usage?

What are the stats of the districts that participated in the studies?

Two of the districts are located in Pennsylvania, and one is in Arizona. Though not intentionally selected for this reason, the three districts that participated in the Study Island research projects share some similarities. All three districts are moderately large, with two districts enrolling approximately 17,000 students in grades K–12 each and the third district enrolling 10,000 students in grades K–8 only. The three districts  historically performed at levels lower than the state average on the end-of-year state tests. Additionally, all three districts had student populations who were majority Hispanic, ranging from 67 percent to  83 percent of the overall student body, a significantly higher percentage than the overall Hispanic population in the districts’ respective states.

Data

We examined student-level data from the 2016–17 academic year for the three participating districts. The districts provided student-level demographic information and state-test-score data from the previous two years’ testing periods (spring 2016 and spring 2017), and these data were matched to Edmentum’s internal Study Island usage data using student identification numbers.

Findings

Though results vary somewhat by grade and subject, we found that students of similar ability who used Study Island have higher scores on average and are more likely to attain at least a proficient level on their state tests across each of the three research sites. This is particularly the case in mathematics.

ESSA Standards of Evidence

Each of the three research studies were quasi-experimental, meeting the Every Student Succeeds Act (ESSA) criteria for moderate evidence, the second highest level of evidence delineated through the ESSA legislation. More information about ESSA evidence standards can be found here.

What’s next for Study Island research?

Edmentum continues to invest in high-quality research to determine the effectiveness of its programs. We are excited to continue more research on Study Island during 2019. Our plans include the following:

1. In these three highlighted research studies that were conducted during 2018, a Study Island user was defined as any student who used Study Island at any level, even in very low amounts. Going forward, we are modifying our operational definition of a Study Island user to consider how many of the Study Island topics a student worked on over the course of a school year. Each Study Island topic is aligned to state standards, and these are the standards on which students are assessed in high-stakes, end-of-year state tests. By examining the number of topics that a student has worked on, we can better understand how much of the standards-aligned content that the student had a chance to practice on. This will help us focus even better on implementation best practices in Study Island, and it is a better definition of the mechanism that would link Study Island usage to state-test achievement.

2. In the first quarter of the year, we expect to release a third-party, independent research study that will investigate how student outcomes are associated with using Study Island.

3. There are several other research sites across the country with which we plan to continue investigating how Study Island is associated with student outcomes on state tests.

For more information about Study Island and how you can use it to drive student outcomes, check out our website and find out why 24,000 schools in the nation partner with us!