What the above statement means for the current OnDataSuite 2022 Accountability A-F reporting tools

We have been working on updating the A-F system reports and we are ready to receive the agreed upon file formatted files once they are released to districts and start calculating the A-F section of the system using the most recent manual release data. The only thing we are waiting on is the FINAL TEA manual release to confirm any undisclosed / updated calculations that may need to be written into this years reports.

Files you will need to load to calculate the 2022 A-F preliminary reports
Below are the list of data files you will need acquire and load into your OnDataSuite file center locations as we start to calculate the predicted A-F reports.

Required Cambium Assessment Files

• STAAR/STAAR ALT 2020 Fall EOC (Dec) – 2021-2022 Data File Format | Administration Date 1321
• STAAR/STAAR ALT 2021 (Summer) – 2020-2021 Data File Format | Administration Date 1621
• STAAR/STAAR ALT 3-8 – 2021-2022 Data File Format | Administration Date 0522
• STAAR/STAAR ALT 2021 EOC (Spring) – 2021-2022 Data File Format | Administration Date 1522
• TELPAS?TELPAS ALT 2022 – 2022 Data File Format | Administration Date 0322

Important note about Cambium Files:
Please make sure that the files loaded adhere to the state required data file format. If last years December EOC files are any indication, Cambium will be releasing these files directly into the District Testing Coordinators (DTC) TIDE inbox. If any of the files listed below are not formatted correctly, the reports will not be able to run the data. Remember after loading your assessment files to run reports under their independent reporting locations in OnDataSuite to determine reasonableness in your data sets.

Important note about loading files into correct location:
The files listed above must be loaded into the OnDataSuite file center locations under the banner titled (Cambium / ETS / Pearson)  DO NOT LOAD these files under the SDR/SIRS location. If you have done so, you will need to put in a ticket to have the files removed so that you can re-upload them into their correct location. If you load your files into both locations this will cause issues. 

• 2021 Fall TSDS – Needed because released files did not include Fall County District Campus Number (CDCN)
• 2022 Fall TSDS

Required TEA – TEAL released reports

• Accelerated Testers
• Final Cohort Graduation files:
  • Four-Year Longitudinal Information
  • Five-Year Extended Longitudinal Information
  • Six-Year Extended Longitudinal Information
• CCMR Verifier
  Something new ‘planned’ but not yet tested will be the CCMR section of this years accountability. In prior years we have left the values as calculated using the early warning system calculations, but this year we intend to use the actual TEA CCMR Verifier file.

Changed AEA outcome bonus labels to reflect when you did not meet minimum size requirements.

Updated SAT/ACT inclusion calculation that was not including A1 or BI students

Adjusted Domain 3 English Language Proficiency Status calculation to include in the numerator Advanced high students that did not have TELPAS Alternate Yearly Progress Indicator.

Fixed issue in CCMR Calculation that was leaving out college Prep indicators . This effects both CCMR EWS and CCMR Portions of the A-F

Calculating for Non-Continuous Enrollment fixed on CCMR using TEA CCMR File

Added 2020 Graduation Rates to Closing the Gaps – Federal Grad Rates Status to be used in new Grad Rate Performance Target calculations

Changed domain ratings. They have been adjusted for Senate Bill 1365 to reflect the No Rating label “NR” alongside of the calculated rating. The will be displayed as follows NR (D)

From 2022 Accountability Manual (07/07/22)
Chapter 5 Calculating 2022 Ratings, Page 45
Overview
In 2022, districts and campuses receive a A–C ratings or a Not Rated: Senate Bill 1365 label overall and in each domain. This chapter describes the process used to determine the ratings for districts and campuses.

https://tea.texas.gov/sites/default/files/chapter-5-2022-calculating-ratings.pdf

Drill downs for Domain 1 CCMR Actual was pulling an incorrect list. We have disabled this until we can fix this. Until this is fixed, districts can verify their actual CCMR counts from the CCMR Verifier report for year 2020-2021 located under the CCMR – College, Career & Military Ready >TEA Tracker/Verifier> CCMR Verifier report

Due to assessment files omitting Fall PEIMS County District Campus Number from TELPAS ALT  test files,  Reports will be populating this from your Fall PEIMS TSDS snapshot Campus records.

Domain 3 EL Proficiency Status, updated calculation to include 0’s in for Yearly Progress Indicator in the denominator and Removed them from the numerator.

Fixed issue that was grading Districts that were an A as “NR”

Domain 2 School Progress calculation. There was an issue not counting test correctly for some districts. 

Fixed an issue on Domain 2 for certain districts that Final Letter grade was calculated incorrectly

Adjusted Senate Bill (SB) 1365 Logic to report “NR”

Late afternoon updates

Fixed issue in Domain 2 if district did not have a rating for Academic Growth it was incorrectly calculating final score

Domain 3 ELP only requires TELPAS 2022 file for Calculation now.

Due to Cambium files omitting Fall PEIMS County District Campus Number from STAAR EOC December test files, Reports will be populating this from your Fall PEIMS TSDS snapshot Campus records. This is only being done to the December file as a temporary fix until an updated file is made from Cambium.

Updated Domain 3 – Graduation Rate Status (Federal) component – Eco Dis calculation corrected
(TEA changed the file for eco dis from 0 and 1’s to Y, N, and 9 so we had to go back in and adjust items coded for prior years)

Domain 2 Summary now populating with both academic performance and relative performance and choosing the highest value of the two. This was commented out from last year due to covid and had to be recoded into this years calculations.

Methodology is unchanged from the 2020 Accountability Manual except for the removal of small numbers analysis.

Fixed issue on Domain 3 Closing the Gaps CCMR Performance section was not reporting numerators

Fixed issue on Domain 3 Summary – Student Achievement Domain or Federal CCMR was not displaying & Met and Total Points

Fixed issue with drill down totals not matching aggregated link values

Fixed an Issue on Domain 1 Summary that was including students that did not meet the minimum size requirement for A-F . They are now excluded in the list.

Added new column in Domain 1 Summary Student Drill down “Under Min Size Req” This is Either a Y or N

We are currently updating the closing the gaps Domain 3 section

Added target goals that align to latest TEA 2022 Accountability Manual as of 06/17/2022. If targets get updated by final release of manual we will have to go back in and make adjustment

https://tea.texas.gov/sites/default/files/chapter-4-2022-closing-the-gaps-domain.pdf

A note about the 2022 A-F Predicted results that will be coming out soon

Just a reminder.  The process of calculating A-F each year involves multiple items that can affect the final calculation of the results, and as such can take some time on our end to make sure that the reported PRELIMINARY A-F predictions are as accurate as possible.

Please remember that OnDataSuite programmers can program our reports in accordance with TEA guidance from the latest manual specifications,  but until we have all data files in from several districts, there is no way for us to determine how accurate our A-F reports are until we have run them for multiple districts and district feedback via the ticket system lets us know they are as accurate as possible.

As the first few days of data files start to come in and get calculated, we rely our YOU, our district users, to help us determine if our reporting is accurately reporting for the current year. Please read below and then if you have issues this year, create a ticket and make sure to provide as much detail as possible when describing why you feel the report is incorrect. A good example would be if your ticket gives specific student examples that should or should not be included in counts.

1. Our calculations are accurate? 
   We are calculating the results using the methodology provided via the TEA 2022 Accountability Manual. However, if the files you are uploading have issues from the vendor, or if the file is missing, then the calculation will not be accurate.  If the assessment file you uploaded was replaced by the vendor with a new file, then you may need to request that the original file be removed.
2. Is all of  your data uploaded coming from a valid file?
   Please make sure that ALL required files have been uploaded. This happens each year. A district will need to verify that every EOC, 5&8 and the 3-8 STAAR files are loaded as well as your TELPAS and final graduation rate (4,5,6 year) files.  This year we are asking for CCMR Verifier & Accelerated Tester files.
3. Did you review your data for data reasonableness?
   After you load a file, please utilize the Assessment, reporting features in OnDataSuite to determine that the files you loaded make data sense. If you know you have approximately 300 EOC A1 testers and you only are seeing 100, that is a clear sign that the uploaded files are lacking. Additionally, make sure that the numerator and denominator values being used to calculate your A-F reports are accurate. Do this by drilling down into the student lists and making sure all your kids are there. This can be done first with the smallest sample (like pick on campus) versus trying to determine reasonableness on the grand scale.

June 1, 2021

1. Since TEA has said there will be no overall domain or scale scores we are determining how this will be displayed. It currently displays letter grades but this should not considered a true letter grade for Domain 1.
2. CCMR was updated to exclude two previously used indicators:
   1. Enlist in the U.S. Armed Forces(pending data receipt)
   2. Complete CTE coherent sequence coursework and earn credit aligned with approved industry–based certifications

June 1, 2021

As per TEA guidance, Part A: Academic Growth will not be calculated for 2021.

June 1, 2021

Academic Growth Status
Closing the Gaps: Academic Growth Status will not be calculated for 2021

Federal Graduation Rate:
We are still working on updating this calculation to adjust it for the 2021 year.

Methodology updated. Indicator may be met with 3 options:

1. Meet long–term target and improve at least 0.1% over baseline rate

2. If #1 is no, meet interim target and improve at least 0.1% over prior year rate

3. If #1 and #2 are no, meet growth target

Archive selections below are kept on file to serve as point of reference for prior year changes

July 2, 2019

Domain 1 – (CAMPUS) Forcing test records for STAAR campus to match fall_peims_co_no to not be counted for campus. STAAR student records will only be counted if campus match fall_peims_cdcn_no.

July 1, 2019

Domain  1 – Added Actual TEA CCMR back to calculation after the release of the file from TEA

June 25, 2019

Domain 1 – Updated AEA graduation rate to include in the numerator
(Number of Graduates + Continuers + TxCHSE Recipients in the Class ) 

Domain 1 – AEA Annual Dropout Rate – Conversion The annual dropout rate conversion is also modified for AEA campuses and districts.
100 – (grade 9–12 annual dropout rate x 5) with a floor of zero By using the multiplier of 5.

June 24, 2019

Domain 1,2 & 3 – Removed year dropdown.

Domain 1,2 & 3 – Added search for District/Campus dropdown

Domain 1,2 & 3 – Added District/Campus number and name to all detail reports title

June 21, 2019

Domain 1 – Raw to scale conversion adjusted for AEA designated district values

June 18, 2019

Domain 1 – (CAMPUS) Forcing test records for STAAR campus to match fall_peims_co_no to not be counted for campus. STAAR student records will only be counted if campus match fall_peims_co_no .

Domain 1 Achievement (CAMPUS) – Updated Scaling calculation

June 18, 2019

Updates made to EL inclusion/exclusion will affect overall student performance levels and in most cases should decrease the total denominator of testers while increasing the raw score value

• Added STAAR Component—Inclusion of English Learners ELs who are year one in U.S. schools are excluded from accountability performance calculations.
• Added ELs who are in their second year in U.S. schools are included in accountability for 2019 .
• Added ELs who are in their second year in U.S. schools are included in the STAAR component using the EL performance measure.
• Added ELs who are in their second year in U.S. schools who have a parental denial for EL services do not receive an EL performance measure.
• Added STAAR Alternate 2 assessment results are included regardless of an EL’s years in U.S. schools.
• Added Unschooled asylees, unschooled refugees, and students with interrupted formal education (SIFEs) are not included in state accountability until their sixth year of enrollment in U.S. schools.
• Added Grad Status =1 to display for student Achievement CCMR display not the calculation  and Domain 3.

June 14, 2019 

• Fixed Scaling Letter Grade
• Fixed Grad Rate Calculation
• Change Grad Rate Calculation to exclude districtRateOnly = 0 from TEA data
• Change display for Elementary and Middle school to display only STAAR component

June 11, 2019

Summary report values are defaulted to show the calculated ODS values and not TEA CCMR upload values until the final TEA CCMR file is available.

This is due to the fact that the first two CCMR preview files TEA provided were not populated with all the data indicator field values.

Districts CCMR ODS value is calculated via PEIMS coding as well as any external data sources such SAT, ACT, AP/IB and  TSI that result in point accumulation. 

Detail report fixed Cohort Grad Rate display
The Student Achievement Detail report now includes the cohort grad rate in the display

July 2, 2019

Domain 2 – (CAMPUS) Forcing test records for STAAR campus to match fall_peims_co_no to not be counted for campus. STAAR student records will only be counted if campus match fall_peims_cdcn_no.

June 25, 2019

Domain 2 – Fixed issue with drill down for student list.

June 24, 2019

Domain 1,2 & 3 – Removed year dropdown.

Domain 1,2 & 3 – Added search for District/Campus dropdown

Domain 1,2 & 3 – Added District/Campus number and name to all detail reports title

June 20, 2019

School Progress (Domain 2) – Update Relative Performance Scale scores to have a floor (F) rating. These were not in the TEA document for School Progress, Part B: Relative Performance Lookup Tables – (Chapter 5—Calculating 2019 Ratings). This would cause no rating to be returned if it was not in the table provided by TEA.

Domain 2 – (CAMPUS) Forcing test records for STAAR campus to match fall_peims_co_no to not be counted for campus. STAAR student records will only be counted if campus match fall_peims_co_no .

June 18, 2019

Student Growth and Domain 2 and 3
This update will affect student progress measure calculations prior to this date

• STAAR 3-8 parser was not importing a default setting of “S” for prior year test versions.  The lack of this element was preventing matching the process from finding any highest outcomes.  Which intern was preventing these students from being properly included/calculated.

July 3, 2019

Domain 3 – CCMR TEA – Adjusted numerators to include a sum of CCMR. 

Domain 3 – Added Summary.

Domain Overall Summary Added

July 2, 2019

Domain 3 – (CAMPUS) Forcing test records for STAAR campus to match fall_peims_co_no to not be counted for campus. STAAR student records will only be counted if campus match fall_peims_cdcn_no.

Domain 3 – CCMR TEA – Adjusted numerators to include a sum of CCMR.

July 1, 2019

Domain 3 – Fixed overall calculation weights issue.

Domain 3 – Added AEA EOC Retest bonus points for district and campuses

Domain 3 – Student Growth, fixed EL (Current and monitored) Numerator and Denominator

Domain  3 – Added Actual TEA CCMR back to calculation after the release of the file from TEA

June 27, 2019

Domain 3 – Fixed overall calculation weights issue.

Domain 3 – Added AEA EOC Retest bonus points for district and campuses

Domain 3 – Student Growth, fixed EL (Current and monitored) Numerator and Denominator

June 26, 2019

Domain 3 – Fed Grad Rate –  Fixed an issue that was allowing not meets under minimum size to be evaluated.

Domain 3 – Overall Calculation now using – Student Achievement Domain Score: STAAR Component Only (Elementary and Middle Schools) even if its a zero. This was causing it to not be used in the overall calculation 

Domain 3 – Overall Calculation – Fixed an issue where it was using STAAR component instead of CCMD component for overall calculation 

Domain 3 – Add Closing the Gaps AEA Graduation Plan Bonus Points for AEA districts and campuses

June 25, 2019

Domain 3 – Fixed issue causing –School Quality or Student Success (CCMR or Student Achievement) to always default to CCMR (ODS) as its chosen component for final Closing the Gaps calculation.

June 24, 2019

Domain 3 –  Fed Grad Rate –  Fixed issue where indicators that  meet for meeting the targets were not counted.

Domain 1,2 & 3 – Removed year drop down.

Domain 1,2 & 3 – Added search for District/Campus drop down

Domain 1,2 & 3 – Added District/Campus number and name to all detail reports title

June 21, 2019

Domain 3 Academic Achievement (Percentage at Meets Grade Level or above) – Fixed EL Current And Monitor issue. 

June 20, 2019

Closing the Gaps – Graduation Rate Status (Federal) – (CAMPUS LEVEL) Fixed Graduation rate target. Student groups that are at or above 90 percent are required to exceed that rate by at least a tenth of a percent in the following year(s).

Domain 3 – Fed Grad Rate – Added “EXCLUDED GRADUATE” to the numerator

Domain CCMR ODS – Fixed issue causing Earned credit for a college prep course not to be excluded.

June 19, 2019

• Closing the Gaps – Graduation Rate Status (Federal)Note about this change item
  The following update is under review and we are seeking clarification from TEA to determine if the Federal Graduation Rate used in the 2019 accountability will be measured against the past year (2017) grad rate to look for a tenth of a percentage or if it is measured at the 90% as stated in the manual. A review of the 2018 Accountability manual states a similar measurement requirement for class of 2017 graduates (Page 30 of the 2018 Acct. Manual) so we implemented this change AS WE CURRENTLY UNDERSTAND IT NOW.  It is possible that this value will be reverted back to measurement at 90% after we have obtained clarification.Fixed Graduation rate target. Page 36 of the 2019 Accountability Manual states ‘Student groups that are at or above 90 percent are required to exceed that rate by at least a tenth of a percent in the following year(s).’Prior to this update all students and sub populations were measuring 2018 graduation rates against the 90% grad rate. This adjustment/update reviews the districts prior year grad rate and looks for a tenth of a percentage gain in order to be considered as having met the target requirement.
  Example: HanSolo ISD’s all student grad rate in 2017 was 91.5%. This is 1.5% over the 90% mark. In 2018 HanSolo ISD must obtain a graduation rate of 91.6% or greater in order to meet the requirement.2019 Accountability Manual Four Year Grad Rate Target Requirements Used
  
• Student Achievement – CCMR (ODS) – Fixed Drill down issue  
• CCMR (ODS) – Redesigned logic for better accuracy.

June 18, 2019

Updates made to EL inclusion/exclusion will affect overall student performance levels and in most cases should decrease the total denominator of testers while increasing the raw score value

• Added STAAR Component—Inclusion of English Learners ELs who are year one in U.S. schools are excluded from accountability performance calculations.
• Added ELs who are in their second year in U.S. schools are included in accountability for 2019 .
• Added ELs who are in their second year in U.S. schools are included in the STAAR component using the EL performance measure.
• Added ELs who are in their second year in U.S. schools who have a parental denial for EL services do not receive an EL performance measure.
• Added Grad Status =1 to display for student Achievement CCMR display not the calculation  and Domain 3.

Student Growth and Domain 2 and 3
This update will affect student progress measure calculations prior to this date

• STAAR 3-8 parser was not importing a default setting of “S” for prior year test versions.  The lack of this element was preventing matching the process from finding any highest outcomes.  Which intern was preventing these students from being properly included/calculated.

June 14, 2019

Student Achievement Domain Score: STAAR Component Only

• Added Average of Approaches , Meets and Masters. This is now used as the met standard
• Updated Met Performance targets to align to guidance
• Was using the “All students are evaluated if there are 10 or more assessments” for min size requirements. I have changed this to use “Student groups are evaluated if there are 25 or more assessments.”
• Fixed issue where Total Evaluated Indicators were not being counted properly

June 11, 2019

• Component Calculations weights fixed
  Academic Achievement (Percentage at Meets Grade Level or above)
• Fixed issue with SPED (current) drill down
• Graduation Rate Status (Federal)
• Fixed issue causing inflated numerator and denominator values
• Updated calculation
• English Language Proficiency Status
• Fixed calculation and drill down for numerator

Throughout the course of the upcoming accountability cycle, districts may bring items to our attention that require immediate changes to certain areas within the A-F Accountability sections. We value your input and will always attempt to correct any issue as soon as we become aware of what is happening and can locate and confirm official documentation that supports any change. While these types of unscheduled updates are not ideal, we feel that it is important to provide you with results that are as accurate as possible.

This page will provide up to date information on any changes made to the A-F reporting sections located in the OnDataSuite Accountability section.