Fairness and Equity Considerations When Creating and Administering Assessments

Shane Dennison, Edmentum’s National Assessment Consultant, kicks off a new blog series on assessments.

Assessment: the act of assessing; appraisal; evaluation (Dictionary.com, 2013).

High-quality items closely measure the intended learning objective and lack ambiguity (Edmentum, 2012). There are several barriers to meeting this objective, including ambiguous statements, excessive words, difficult vocabulary, unclear instructions, and bias (Haladyna 2004).

Edmentum practices and recommends the following “quality guidelines” for all item types:

• Use novel material to test higher level learning
• Avoid trivial content
• Avoid irrelevant information (window dressing)
• Avoid trick items
• Meet the readability needs of the audience
• Vary the location of the correct answer

In regard to selecting equitable, fair and Common Core appropriate reading passages, Edmentum closely considers and recommends the following criteria:

• Genre
• Source
• Quality
• Content
• Lack of Bias
• Gender Appropriateness
• Diversity
• Length
• Suitability for Reporting Categories
• Reading Level
• Graphics

When judging test items to make sure the test versions of Edmentum Test Packs are comparable, there are four major areas our development team addresses:

• Cognitive level
• Readability
• Plausibility of answer options
• Visuals

If items are comparable in these four areas within a reasonable margin, then the items have been considered comparable for the purposes of generating comparable test versions. To make these judgments, developers use a series of guidelines for each area. The guidelines are not enforced word-for-word in all cases. Instead, they describe an ideal level of comparability at a micro-level. The characteristics of an item as it relates to each of these guidelines are then factored into higher level judgments about the four major areas of comparability. In addition, the four areas have relative levels of importance. For example, more importance is placed on cognitive level and readability than on visuals when making a comparability judgment about an item. In continually developing new tests which mix new items written specifically for the new tests with quality items from our existing database, we find tension between comparability and quality criteria. We develop new tests with an eye on always improving item quality, keeping an awareness that comparability between existing items and new items may not be optimal in order to write the highest quality new items we can provide (Edmentum, 2012).

The following characteristics apply to a test as a whole, rather than to individual test items.

The table that follows describes several characteristics of test items and Edmentum’s procedure for evaluating each characteristic in order to ensure that each new item is comparable to peer items on a test.

 Guidelines for Test Item Comparability