The #1 Curriculum and Assessment Partner for Educators

Looking for Online Solutions?

See Our Products
www.edmentum.com

Two Exciting Research-Based Enhancements to the Exact Path Diagnostic

Two Exciting Research-Based Enhancements to the Exact Path Diagnostic

Our Research team is always working to provide cutting-edge enhancements to Edmentum’s learning and assessment tools. For the 2020–2021 school year, there are two exciting upgrades now available for the Exact Path diagnostic! These enhancements come just in time for the flexible testing environments this school year requires and are important for improving in-school and at-home administrations.

Shorter Diagnostic Administrations

You might notice that students complete the Exact Path diagnostic faster this school year than they did in previous years. That’s not a coincidence! Using modern methods in psychometrics, we implemented a behind-the-scenes change to the way the Exact Path diagnostic algorithm works so that students see up to 20% fewer items while still maintaining the same level of precision in the Exact Path scale score. Younger learners will see the greatest reductions in testing time, though learners across all grades will have reduced test lengths. This means you can spend less time testing and more time learning without compromising the validity of score interpretations. Educators will receive the same quality score reports as in previous years, and students will not see any changes to the interface of the diagnostic.

What exactly changed in the algorithm to achieve the shorter test?

The changes to the algorithm focused on how student scores are updated after each test item is answered. In the diagnostic both in previous years and moving forward, a student’s score is calculated behind the scenes after each item response. This score is called the updated ability estimate. Once the updated ability estimate reaches a targeted level of precision, the test stops. The new scoring algorithm uses a more intelligent approach by incorporating information we know about the student population.

For example, previous diagnostic scores from third-graders in math give us an idea of what typical third-grade math scores will be on the diagnostic, just as a student’s previous score gives us a good idea of what their next score will be. The updated scoring algorithm incorporates this information into each updated ability estimate to more quickly arrive at an estimate of student ability within the targeted level of precision. This method, known as Bayesian estimation, is used in many other fields beyond educational assessment. For example, if estimating how much a home will sell for, that price can be better estimated by incorporating information known about prices of other houses in the neighborhood as opposed to just considering factors of the individual house such as the number of bedrooms. Even though the new algorithm uses information about typical student scores, the test will still adapt appropriately for all students regardless of how above or below average the student might be.

 

Recommended Reviewing Alert

Another exciting new feature is the Recommended Reviewing Diagnostic Alert. Have you ever wondered if some of your students rush through the diagnostic by clicking random answers as fast as possible? This alert will notify educators when that may be the case. When students do not provide effortful responses their scores may be unusually low, resulting in mismatched placements onto the learning path. This feature is even more critical for students that may be testing at home due to school closures because educators cannot observe student testing behavior during the diagnostic like they can at school.

How did Edmentum research scientists determine when to trigger the alert?

The alert triggers when a student spends 5 seconds or less per question on average. The 5-second threshold was set by investigating typical response time (i.e., the number of seconds a student spends responding to a question) to thousands of items across millions of test events from all grades and subject areas. By looking at how many students respond to items in 1 second, 2 seconds, 3 seconds, etc., we identified clear patterns that indicate non-effortful responses typically occur in 5 or fewer seconds. Then, we validated this threshold by examining the percentage of correct responses to responses flagged as non-effortful. If a student is guessing, we expect the student to get the question correct about 25% of the time (for multiple-choice questions with four answer choices) whereas accuracy rates for questions not flagged as guesses are around 50%, which is the goal of the adaptive test. The alert identifies the most egregious cases of non-effortful responses where it is extremely likely the student did not demonstrate their best effort.

The alert shows up in several places within Exact Path with a red warning and will prompt educators to review the diagnostic results to make a decision about how to proceed: reset the diagnostic so the student can test again (if the testing window is still open), adjust the learning path to better reflect where the student is ready to learn, or ignore the alert. Want to see this feature in action? Check out this short video. You can also refer to the Help Center for more how-to assistance.

When the alert is triggered, the educator will see a notification like the below.

Exact Path Notification

Upon clicking on the notification, the following three options will display, making it easy to monitor test validity and take action with just a few clicks.

Action Options Exact Path

Together, these two research-driven diagnostic features reflect our ongoing commitment to ensuring your Exact Path diagnostic scores are valid and students are put on the right track toward meeting their personal growth goals. Still looking for more information on administering assessments successfully this year? Check out our blog, 5 Tips for Administering the Exact Path Diagnostic.

audra.kosh's picture

Dr. Audra Kosh began her career in education as an eighth-grade math teacher. After transitioning out of the classroom to pursue her passion for research, Audra completed a Ph.D. in Learning Sciences with a focus on educational measurement and mathematics education at the University of North Carolina at Chapel Hill. Now working as a Research Scientist at Edmentum, Audra does psychometric analyses and assessment research for Edmentum’s suite of assessments.