What Can We Learn from the First Round of Federal Feedback on ESSA State Plans?

Wednesday, August 16, 2017

Back in April, the first group of 17 states submitted their new ESSA accountability plans for review by the U.S. Department of Education (DOE). By design, ESSA has very specific requirements for state plans in some areas, like what subjects and grades are tested. But in other areas, like setting long-term goals for student achievement and rating school quality, states are given freedom to interpret the legislation and create their own plans. The looming question as states worked through the process of finalizing plans has been how will federal reviewers interpret the states’ work? Will plans get “rubber stamp” approval? Or will the DOE review these plans with a fine-tooth comb, and ask states to make significant changes? We now have some answers to these questions.

The Department of Education has made public the first set of feedback on submitted state plans, and a much clearer picture of what expectations for accountability every state will have to meet has emerged. In fact, on August 2nd, Delaware became the first state to not only receive feedback, but to respond with changes and receive final approval on their state accountability plan. As the picture of state accountability becomes clearer, and as educators prepare for the start of the 2017-18 school year, it’s a good time for educators to get familiar with their states’ accountability plans and understand how learning outcomes and school quality will be measured and reported in the upcoming school year. If you need a refresher on the basics of ESSA before diving into the state plans, check out this quick podcast:

What the State Plans Look Like

Of course, every state plan is unique, and it’s important for every educator to put in the time and effort to get to know the ins and outs of their own state’s submission. But, to help point out important things to watch for and provide some insight to the varying approaches states are taking, I’d like to take the opportunity to look at three states—Massachusetts, Tennessee and New Jersey—that have already submitted plans and received feedback from the Department of Education. We’ll focus on six key areas of the plans to compare differences in each state’s methodology and understand how the Department of Education is influencing and approving plans. These six areas are:

Long-Term Goals–each state is required to set challenging, long-term student achievement goals for the future.
Academic Indicators–each state must define how they will measure academic achievement and progress.
High School Graduation Rates–each state must define how they will calculate and weight graduation rates in their system.
School Quality Indicators–each state must include at least one non-academic indicator as a measure of the environment of learning within a school.
English Language Proficiency-states must define how they will measure and track English language learners’ (ELLs) progress towards proficiency in English.
Rating School Performance–states must define how they will weight and calculate each accountability indicator to report school performance.

Long-Term Goals

Massachusetts’ plan–aims to reduce the proficiency gap for all students and all subgroups by one-third by 2022. Because Massachusetts transitioned from the PARCC assessment to a new state test (The Next-Generation MCAS) last year, baseline data to build annual projections is not yet available. The state plans to increase the graduation rate from 87.3% to 91% in 2020. Massachusetts also proposes to increase the percent of ELL students meeting proficiency standards from a 61.8% baseline to 81% by 2022.

Tennessee’s plan–proposes to reduce by 50% the number of students not achieving the on track or mastered level on the state’s annual assessment by the 2024-25 school year. The state also proposes to increase the graduation rate for all Tennessee students from 85.5% to 95% and achieve a ranking in the top half of state scores on the National Assessment of Educational Progress (NAEP) by 2024-25. For English Language Learners (ELL), Tennessee plans to increase the number of students achieving WIDA annual growth targets from 51.3% to 75% by 2024-25.

New Jersey’s plan–proposes that by 2030 at least 80% of all students and all subgroups will achieve proficiency on the state assessment, PARCC. This would require a significant increase in test scores from a baseline in 2015-16 of 41.9% in math and 51.5% in ELA.

Academic Indicators

Massachusetts’ plan–will weight academic indicators at 85% of their overall accountability system as measured on the MCAS grades 3-8 test. An average scale score will count as 60% (academic achievement) while progress (based on Student Growth Percentiles or SGP) will account for 25%.

Tennessee’s plan–will weight academic indicators at 80% of the overall accountability system, with proficiency counting for 45% and annual growth counting for 35%. Annual growth for ELLs will count for 10%.

New Jersey’s plan–will weight academic indicators at 70% in their proposed accountability system with proficiency at 30% and growth (SGP) at 40%.

High School Graduation Rates

Massachusetts’ plan–proposes using three data points to measure graduation rates: four-year cohorts, five-year cohorts, and annual dropout rates. These combined indicators will get a weighting of 17.5% on Massachusetts’ accountability plan.

Tennessee’s plan–weights graduation rates at 25% of the overall accountability score tracking annual progress towards meeting the state’s long-term graduation goals.

New Jersey’s plan–proposes using both four-year and five-year graduation rates in their system weighted at 40% of accountability for high schools.

School Quality Indicators

Massachusetts’ plan–includes three indicators to measure school quality: chronic absenteeism, success in grade 9 courses, and successful completion of broad and challenging coursework (Honors, AP and International Baccalaureate courses). These indicators combined will receive a weighting of 7.5% in the state’s accountability system.

Tennessee’s plan–proposes using a chronically out of school measure (absenteeism and suspensions) and ACT/SAT scores and CTE career pathway certifications as the main school quality indicators and gives them a 10% weighting towards overall accountability.

New Jersey’s plan–will use chronic absenteeism as their school quality indicator, which it defines as “absent more than 10% of the school year”. New Jersey will weight this measure at 10% of the total accountability score.

English Language Proficiency

Massachusetts’ plan–proposes using the annual WIDA ACCESS for ELLs assessment growth target to track interim progress for all ELL students. In addition to tracking annual growth rates for ELLs, Massachusetts will look at student attainment of English language proficiency. Together these indicators will be weighted at 5% in schools with measurable ELL populations.

Tennessee’s plan–will also measure ELL growth using the WIDA recommended annual growth targets on the ACCESS assessment. ELL annual growth will be weighted at 10% of the state’s accountability score.

New Jersey’s plan–like Massachusetts and Tennessee, is using annual growth measured on the WIDA ACCESS for ELLs to track progress towards English proficiency. ELL growth will count for 10% of the total accountability score in New Jersey.

Reporting Data and Rating School Performance

Massachusetts’ plan–uses a combination of both normative and criterion-referenced methodology to place schools into a 6-tiered ranking system based on a 100-point scale.

Tennessee’s plan–proposes using an A-F grading system with schools receiving a grade of D or below designated as Focus Schools that will receive targeted support from the district and state.

New Jersey’s plan–for rating schools is not a simple letter grade or cumulative calculation. New Jersey is proposing converting each accountability indicator in their system to a percentile ranking, combining these percentiles into a total percentile, then placing schools into three categories: Met Target, Above Target, and Below Target. New Jersey schools will receive a percentile score on a 100-point scale and placement into one of three categories based on this score.

What the DOE feedback is saying

Federal feedback on Massachusetts’ plan:

Massachusetts uses a scale score to calculate academic achievement. While ESSA allows scale scores to be used to calculate academic progress or growth, it requires academic achievement to be reported as grade-level proficiency.
The state’s plan includes a drop-out rate within its academic indicators. ESSA does not allow drop-out rates included in academic indicators, but would allow this as a school quality indicator.
The plan includes a quality indicator base on the percentage of students who successfully complete “broad and challenging coursework,” such as Advanced Placement, honors classes, and International Baccalaureate coursework. ESSA requires equal access to advanced courses (all students in all schools) if this is to be included in the state’s accountability index.
Massachusetts plans to identify the lowest 5% of schools among all schools. ESSA requires that states identify the lowest performing Title I school.
Massachusetts has not clearly defined the exit criteria for schools identified for targeted support.

View Massachusetts’ complete plan and Federal feedback here

Federal feedback on New Jersey’s plan:

New Jersey proposed allowing all middle school students to take a high school mathematics course and would exempt these student from the state math assessment. Federal guidance suggested only 8^th grade students taking a high school math course should be exempted from the 8^th grade state math test.
New Jersey’s plan proposed treating ELL students differently in calculating high school graduation rates. Federal reviewers made it clear that all students must be treated the same when tracking graduation rates.
New Jersey’s plan was criticized by Federal reviewers for not including ELL progress data in the identification of schools for comprehensive support in the initial year of the plan’s implementation. New Jersey planned to begin including ELL progress in 2018-19.
New Jersey’s plan only considers consistently underperforming subgroups in graduation rate and academic achievement indicators. Reviewers made it clear that all indicators must report on all subgroups and identify consistently underperforming subgroups.

View New Jersey’s complete plan and Federal feedback here.

Federal feedback on Tennessee’s plan:

Tennessee’s plan explicitly defines the state as an “English-only” state, and therefore does not identify or define other languages present in the student population as required by ESSA.
The plan includes a subgroup that combines Black/Hispanic/Native American (BHN). ESSA requires reporting data separately for each subgroup.
While the state plan provides both a four year and extended year graduation rate, it only plans to report interim progress on the four-year rate. ESSA requires setting long term goals and reporting interim progress on all accountability indicators.
Tennessee plans to identify the lowest performing 5% of schools among all schools. ESSA requires that states identify the lowest performing Title I schools specifically.
The state plan uses both a 4 and 5-year graduation rate, but does not clarify which graduation rate it will use to identify schools that fail to graduate one-third or more of their students.
Tennessee’s plan does not clearly define “consistently underperforming” so it is unclear how the state will identify schools with consistently underperforming subgroups.

View Tennessee’s complete plan and Federal feedback here.

Overall, the submission and review of state accountability plans is on track with the implementation timeline initially laid out when ESSA legislation was approved. The feedback from Federal reviewers so far can mostly be described as technical compliance with ESSA requirements, rather than indicating major disconnects between state plans and Federal expectations. Most states are fully meeting requirements in areas where ESSA offers the most clarity, like assessment of math, ELA, science, and English language proficiency. Areas of state plans where ESSA provides less direction, such as regarding school quality indicators and tracking subgroup progress towards closing achievement gaps, are receiving the most feedback from Federal reviewers at this early stage.

This initial round of state plan submission and federal review has left us with valuable insights as to what ESSA implementation will look like in schools across the country. Look for another update later this fall after the second window for state plan submission in September. In the meantime, learn more about your own state’s plan by checking out Edmentum’s ESSA & State Accountability Systems hub, or register for our webinar to hear one superintendent's perspective on meeting ESSA requirements first-hand.

Additional resources for your reference: