[Reporting Feature Focus] How Does Exact Path Determine Grade-Level Proficiency?

It is common for the end of the school year to be marked by a state summative assessment designed to determine if students have achieved proficiency for their grade level in subject areas such as math and reading. That important metric, grade-level proficiency, is essential for educators, parents, and school administrators to determine if a student has mastered the knowledge and skills to be successful in later grades or whether the student needs additional support.

Exact Path, Edmentum’s diagnostic-driven, individualized learning program, now includes grade-level-proficiency classifications for students in grades K–8 who take the Exact Path diagnostic. We shared more about this and other 2021–22 enhancements in this recent post. Here, let’s focus in on how Edmentum researchers and educators worked closely together to develop a valid grade-level proficiency indicator that provides educators with helpful information for future instruction.

Grade-Level Proficiency Labels

Edmentum’s research, product, and learning design teams wanted grade-level proficiency classifications to provide useful information about whether students were on track to meet end-of-year grade-level expectations as measured by their state summative tests. We started by reviewing the state summative proficiency labels in the 25 states where Exact Path is most widely used and found that most of the states defined four proficiency categories with language related to meeting grade-level expectations. We then summarized the state-level data by defining our four proficiency levels as:

Level 1: Below Grade-Level Expectations

Level 2: Approaching Grade-Level Expectations

Level 3: Meets Grade-Level Expectations

Level 4: Exceeds Grade-Level Expectations

Next, we transformed summative test scores from our state research into Exact Path diagnostic assessment scores so that we could use student results on the Exact Path diagnostic to make grade-level proficiency classifications. This gave us a rough idea of which Exact Path diagnostic scores corresponded with each of the proficiency levels.

Data-Driven, Standard-Setting Workshops

From there, we wanted to make the classifications more accurate by incorporating expert judgments from educators in a formal standard-setting study. Given the importance of accurately classifying students into proficiency levels, we partnered with an international leader in standard setting, Dr. Daniel Lewis, who codeveloped the widely used bookmark standard-setting procedure and facilitated dozens of standard-setting workshops, including ones for Smarter Balanced and ELPA21.

Dr. Lewis led the design of the standard-setting study and facilitated two weekend workshops involving more than 50 educators who had an average of 15.3 years of teaching experience across 30 states, plus Puerto Rico. The educators served as panelists during the weekend workshops and provided expert judgments about what knowledge and skills a student should have to be classified in each of the four proficiency levels for each grade level (K–8) and subject area (math, reading, and language arts).

To inform these decisions, educators were provided with sets of items (or test questions) that represented a typical diagnostic assessment a student might see in each grade level and subject area. (Note that the Exact Path diagnostic is an adaptive test, so each student receives a test that is personalized for her or his ability level.) Descriptive information was provided for each item, such as the relative difficulty based on previous student performance data and the Common Core State Standards (CCSS) alignment code.

Educators then performed a thorough committee review of each item, discussing what the item measured and the knowledge and what skills are required to respond correctly to the item. Finally, educators were shown which items were located near the proficiency boundaries calculated from the 25 states summative test data, and they classified each item into the appropriate proficiency level. These expert judgments allowed Dr. Daniel Lewis and his team to calculate Exact Path score ranges for each proficiency level, grade level, and subject area. For example, here are the score ranges for math. Grade level is on the horizontal axis, and the Exact Path diagnostic score is on the vertical axis:

Each point in the figure represents the lowest score needed to be classified in the above-proficiency level for a given grade. For example, a kindergarten student must score at least 653 to be classified as approaching grade-level expectations, at least 738 for meets grade-level expectations, and at least 804 for exceeds grade-level expectations. Note that students need higher scores to meet proficiency standards in the higher grades.

Performance-Level Descriptors

The grade-level proficiency feature is first and foremost for our Exact Path educators, and we wanted the information to be useful for them. To ensure that the feature has practical value for educators, subject-matter experts from Dr. Lewis’ team and Edmentum’s learning assessment design team reviewed the detailed item descriptions from the standard-setting workshops and drafted performance-level descriptors (PLDs) for each grade level and subject area. The PLDs describe the knowledge and skills of a typical student in each proficiency level, and they can be used to inform future instruction. For example, here is the meets grade-level expectations PLD for reading, grade 2.

PLDs for each proficiency level, grade level, and subject area are included in the Exact Path Class Results Report.

Educator Feedback

At the end of the standard-setting workshops, we asked educators serving as expert panelists about their experiences and about the usefulness of the grade-level proficiency indicator. Here are a few of their thoughts:

We would like to offer our thanks to our educators for helping us develop a grade-level proficiency indicator! This Exact Path enhancement will offer a valid measure and useful guide as to which students have mastered the knowledge and skills to be successful in later grades and which students need additional support.

Want to learn more about this and other 2021–22 Exact Path enhancements? Join us for a live webinar on what’s new in Exact Path!