[Research] Edmentum Test Packs in Support of ACT Preparation

Student scores on high-stakes assessments are an important part of understanding student learning, as well as essential elements of school accountability under federal and state governments. When schools invest in programs like Edmentum’s Test Packs in Support of ACT® Preparation (referred to as Test Packs for brevity), they want to see student growth in assessment results. In 2018, the Edmentum Research and Design team completed a study that was set in an Oklahoma school district. This district used Test Packs to structure instruction and prepare high school juniors for the ACT test, which is a portion of the state’s accountability system for high school graduation. The district requested support from the Edmentum Research team to examine the relationship between usage of Test Packs and student performance on the ACT test.

What are Test Packs?

Edmentum Test Packs in Support of ACT Preparation is an assessment and curriculum tool that evaluates students against the ACT College and Career Readiness Standards®. Then, it provides an individualized prescription, or learning pathway, based on student assessment results that can be utilized by students in order to gain instruction and practice on standards tested by the ACT test.

Research Questions

The study examined three research questions:

1. What are the trends and patterns in this Oklahoma district’s student usage and performance on Test Packs assessments during the 2017–18 school year?
2. How is performance on Test Packs assessments correlated with performance on the ACT test by subject among the district’s 11th grade students?
3. What are the trends and patterns in student usage on Test Packs prescriptions during the 2017–18 school year?

Data

The district provided student level demographic information and ACT test score data from spring 2018. These data were matched to Edmentum’s internal Test Packs data using student identification numbers.

Findings

Through a series of descriptive and correlational analyses, the findings in this study show that while most 11th graders in the district used the Test Packs assessments over the course of the school year, only a small proportion of students used the associated content designed to prepare students for the ACT test, and among those who did access the content, usage was low.

By calculating the correlation between student scores on the ACT and the Test Packs assessments, we can investigate the relationship between the two tests and provide evidence of predictive validity. Figure 1 shows the relationship between standardized student scores on Test Packs Form 2 and the ACT scores by subject. The black line represents the line of best fit between student scores, and within all subjects, a clear relationship between Test Packs scores and ACT scores is evident.

Figure 1. Relationship Between Student Scores on Test Packs Assessments and the ACT Test by Subject

The results of the correlation between student scores by subject on the ACT assessment and Test Packs Form 2 are shown in Table 1. Because the test scores were not normally distributed, Spearman’s rho was calculated and reported in Table 1 and is interpreted in the same manner as the Pearson correlation coefficient. The correlations for English, reading, and science, ranging from .54 to .63, are considered large associations, while the correlation for math is considered a medium association because it does not quite meet the .50 threshold for a large association.

However, these findings cannot be used to predict individual student ACT test scores. The small sample size that is not representative of either the state of Oklahoma or the United States and precludes a prediction, which would require a comprehensive linking study.

Table 1. Correlation Between Scores on ACT Test and Test Packs Form 2 Assessment by Subject

Interested in learning more about how Edmentum can support your students’ ACT preparation? Learn more about Edmentum Assessments, which provide immediate feedback, paired with targeted instruction to help educators zero in on opportunities for improvement.