About CAEP Annual Reporting
As part of CAEP accreditation requirements, UNC-CH shares data demonstrating the effectiveness of its completers’ instruction on P-12 student learning and development, and completer and employer satisfaction with the relevance and effectiveness of preparation.
Measure 1: Impact on P-12 Learning and Development
Measure 1b: Indicators of Teaching Effectiveness
Measure 2: Satisfaction of Employers – Initial Programs
Measure 2: Satisfaction of Employers—Advanced Programs
Measure 3: Candidate Competency at Completion—Initial Programs
Measure 3: Candidate Competency at Completion—Advanced Programs
Measure 4: Ability of Completers to be hired
Measure 1: Impact on P-12 Learning and Development
As of April 20, 2023, NCDPI (North Carolina Department of Public Instruction) has not released EPP (Educator Preparation Program) performance reports for the 21-22 academic year.
These student growth data align with CAEP Annual Report Measure 1a (Initial): Completer effectiveness. (R4.1) and demonstrate completer impact in contributing to P-12 student-learning growth.
The above graph shows P-12 student growth collected annually through North Carolina Educator Value-Added Assessment System (EVAAS) for beginning teachers (3 or less years of teaching) prepared by UNC-CH. The bars show the percentage of UNC-CH beginning teachers who do not meet, meet, or exceed expected growth for and all beginning teachers in North Carolina public schools.
2017-2021 EVAAS data shows that 80% or more UNC-CH completers meet or exceed growth expectations annually. While the percentage of ratings may vary across years, UNC-CH completers outperform beginning teachers across the state on student growth prior to the COVID-19 Pandemic. In 2018-19, 42% of UNC-CH prepared beginning teachers exceeded growth expectations comparing to 13% at the state level. The trend was disrupted by COVID-19. NCDPI (North Carolina Department of Public Instruction) did not collect P-12 student assessment data for 2019-20. 2020-21 EVAAS data show that 84% of UNC-CH prepared beginning teachers meet or exceed growth expectations, but the distribution of ratings is different compared to previous years, which is caused by the COVID-19 pandemic impact on P-12 student learning.
UNC-CH monitors EVAAS data annually seeking opportunities to make targeted improvement in program areas and their coursework. We also monitor these data as they are a key component in the new accountability model for North Carolina EPPs (Education Preparation Programs). Our use of these data for program improvement post-COVID-19 will differ from their use prior to the pandemic. Of note, we are eager to ascertain the if performance trends for UNC-CH and the state data return to pre-pandemic rates—where UNC-CH has higher rates of teachers meeting or exceeding student growth estimates that the state average—or if the trend will shift.
Data Source: NCDPI EPP Performance Report
Measure 1b: Indicators of Teaching Effectiveness
As of April 20, 2023, NCDPI has not released EPP performance reports for the 21-22 academic year. The data below is from prior years.
These teacher effectiveness data align with CAEP Annual Report Measure 1a (Initial): Completer effectiveness. CAEP standard R4.1 and demonstrates completer effectiveness in applying professional knowledge, skills, and dispositions.
This set of data/graphs shows school administrators’ ratings of UNC-CH prepared beginning teachers (3 or less years of teaching) on the NCEES criteria (North Carolina Educator Evaluation Systems):
- Standard 1: Teachers Demonstrate Leadership
- Standard 2: Teachers Establish a Respectful Environment for a Diverse Population of Students
- Standard 3: Teachers Know the Content They Teach
- Standard 4: Teachers Facilitate Learning for Their Students
- Standard 5: Teachers Reflect on Their Practice
NCDPI collects teacher effectiveness data on beginning teachers each year except for Covid impacted 2019-20. School Administrators rate at the level at which beginning teachers meet Standards 1-5 as they move from ratings of ‘Developing’ to ‘Distinguished’. Effective 2020-21, at the end of their third year beginning teachers must be rated ‘Proficient’ on Standards 1-5 on the most recent ‘Teacher Summary Rating Form’ to be eligible for the Standard Professional 2 license. New teachers are more likely to be rated lower on evaluation standards as they are still learning and developing new skills and knowledge.
2016-17 through 2018-19 and 2020-21 evaluation results indicate that UNC-CH prepared beginning teachers consistently receive high ratings on all NCEES criteria. 98% to 100% of UNC-CH beginning teachers are rated proficient, accomplished, or Distinguished on all NCEES criteria, which is consistently higher than statewide ratings. Specifically, as shown in the radar graph for 2020-21, 100% of principals rated our beginning teacher as more or much more effective in all surveyed teaching practices.
Before the pandemic, a comparison between 2016-17 through 2018-19 teacher evaluation data finds that UNC-CH beginning teachers improved effectiveness over all teacher standards. We consider this an expected result of continuous improvement of our programs. Due to covid-19, NCDPI did not administer the survey 2019-20. The data shows that UNC-CH prepared beginning teaching continues to be rated as 100% proficient or above on all teaching standards.
The program used these data as an important marker of principal/employer feedback on our completers. We triangulate these data with those from the NC (North Carolina) Employer survey to generate a more holistic view of how graduates are perceived by their supervising administrators during their first years of teaching. We see some points of alignment between the data sources and some areas for further investigation. One area for further analysis and action related to Standard 3: Teachers know the content they teach. Principal ratings and the Employer survey differ in their ratings. As a result, the program is actively considering how it examines content knowledge expertise at admission and as developed throughout the program.
Data Source: NCDPI EPP Report Cards
Measure 2: Satisfaction of Employers – Initial Programs
The data for most of this section is from the prior academic year, as NCDPI has not yet published the 21-22 years reports. The only table update is the first graph as this data was provided for CAEP purposes.
NC School Employer survey data aligns with CAEP Annual Report Measure 2 (Initial): Satisfaction of employer and stakeholder involvement represented in CAEP standards R4.2 and R5.3.
First administered in 2016-17 by NCDPI and the Education Policy Initiative at Carolina (EPIC), the NC School Employer survey asks school principals or assistant principals to rate the effectiveness of first-year teachers on 29 teaching tasks that are aligned with the state’s professional teaching standards, comparing to other NC prepared, out-of-state, and alternative entry first-year teachers.
The first-year teachers are rated on a scale of 1 to 5: Much Less Effective, Less Effective, Comparable, More Effective, and Much More Effective. This set of graphs shows principals’ overall satisfaction with UNC-CH first year-teachers, and mean rating scores for each of the five professional standards compared to other EPP groups.
The NC School Employer survey results from 2016-17 to 2020-21 indicate that UNC-CH graduates are highly effective in teaching, and principals are satisfied with our students. Over the past five years, 92% or more of principals reported that UNC-CH candidates performed different teaching tasks comparable, more effectively, or much more effectively than first-year teacher prepared by other programs and states. The 2021-22 data mark the first time UNC completers overall were rated lower than the state average. We are eager to examine standard-level data to identify outliers or trends to address.
Data Source: Workbook: EPP Employer Satisfaction (nc.gov)
Measure 2: Satisfaction of Employers—Advanced Programs
These data are aligned with CAEP Annual Report Measure 2 (Advanced): Satisfaction of employer and stakeholder involvement as outlined in CAEP standards RA4.1 and RA5.3.
To support EPPs’ continuous improvement efforts and accreditation needs, NCDPI started providing employment data for EPP completers in 2018 and set to release this data annually in April. NCDPI employment data identifies all UNC SOE (School of Education) completers who have graduated with a degree and been employed as an educator in North Carolina public or charter schools. These data include completers’ identifications, employee information, completers’ job title, and the degree levels earned at UNC.
Since UNC-CH is using proxy measures to address Employer Satisfaction with our advanced candidates, the unit is actively working to develop and implement stronger feedback mechanisms for its graduates. One action is the planning and development of an SOE
employment database with the 3 years of NCDPI employment data, and plan to expand this database yearly when new data become available.
NCDPI Employment data has related to SOE completion data to add corresponding completion data: completion term, degree completed, academic plan and subplan, Gender, Ethnicity etc. With this data, we track employment by program and cohort, map employment distribution in North Carolina. As this database grows with more years of employment data, we can track our completers’ career transition, job retention, and employer satisfaction.
Highest Employing NC Public School Units of UNC Completers 2020-21
Measure 3: Candidate Competency at Completion—Initial Programs
This set of assessments align with CAEP Annual Report Measure 3 (Initial): candidate competency at completion. CAEP standard R3.3 and shows UNC-CH candidates are meeting program expectations and are ready to be recommended for licensure.
At program completion, UNC-CH initial teacher candidates are evaluated on multiple performance metrics which include NC Licensure Exams, edTPA, NC LEA/IHE Certification of Teaching Capacity and Course Grades to determine qualification for program completion and licensure recommendation.
The Federal Tile II reporting collects Licensure assessment data including Praxis II and edTPA for all licensure areas, Foundations of Reading for Elementary Education, and calculate pass rate annually to demonstrate EPP quality. UNC-CH pass rate has been consistently 95% or above and significantly higher than North Carolina statewide average.
NCDPI monitors licensure Exam pass rates for all NC Educator Preparation Programs through NC Educator Preparation Program Dashboard. UNC-CH pass rates are 20% higher than other NC EPPS as a whole or by licensure areas.
For the past three years, UNC-CH candidates received an average edTPA score of 45 or above, which is 8 points above the passing score and within this same domain, UNC-CH candidates’ edTPA scores were higher than candidates from North Carolina statewide and the national average.
Data Source: Title II Reports
Measure 3: Candidate Competency at Completion—Advanced Programs
This set of assessments align with CAEP Annual Report Measure 3 (Advanced): candidate competency at completion. CAEP standard RA3.4, and shows candidates are meeting program expectations and are ready to be recommended for licensure.
Since North Carolina does not require licensure exam for advanced licensure areas, UNC-CH advanced programs (MEdX-ECIFS, MSA and EDD Education Leadership) evaluate students’ competency through course embedded assessments (ESAs) and collects ESA (Embedded Signature Assessment) data and other program artifacts in our e-Portfolio platform, Taskstream. All ESA rubrics are aligned with corresponding national standards, which are outlined below for each program.
The preliminary ESA data shows that UNC-CH scored 90 or more on most of national stands, with a variation from 87 to 100 among standards. These data provide evidence of candidate competency and an opportunity to identify area/standards for improvement.
Each of the advanced programs has an annual program review cycle during which its program data is reviewed and analyzed to inform program improvement activities. The data generated
by the ESA forms a critical evidence base for this work and are examined holistically with other quality assurance measures, such as exit surveys and employment data.
EDD Ed Leadership: NELP District Level Standards Assessed in EdD Coursework
MEdX
All data presented in this section align to Early Intervention/Early Childhood Special Education (EI/ECSE) standards. At present MEDX-ECIFS Embedded Signature Assessments (ESAs) assess all the seven EI/ECSE standards; ESA data are presented in data tables below.
MSA
All data presented in this section align to NELP standards. The data below shows how our MSA students are performing. The NELP standards are outlined in the EdD section.
Data Source: Course embedded assessment data collected in Taskstream
Measure 4: Ability of Completers to be hired
This set of data aligns with CAEP Annual Report Measure 4 (Initial and Advanced): Ability of completers to be hired in positions for which they have prepared.
The ability of initially licensed teachers to be hired is best reported through the licensure and employment data from NCDPI EPP Performance Dashboard – Initial Employment and Retention .
The ability of advanced program completers to be hired into positions aligned with their programs of study utilizes employment data from NCDPI. For advanced completers we track employment in NC public schools and, more specifically, employment in certain job roles. The exception are graduates from UNC-CH’s MEDX ECIFS (Early Childhood Intervention & Family Support) program. Many candidates return to the agency or private settings upon graduation and, therefore, are not present in NCDPI employment data.
The employment table shows that a high number of MSA and MEdX ECIFS programs are employed by North Carolina Public Schools in their prepared licensure areas one year after program completion. Half of EDD Education leadership graduates served in principal or assistant principal role considered on the route to senior leadership. These data do not track graduates who work in private schools, higher education institutions, or other sectors of educational institutions.
As shown by the employment distribution maps, UNC-CH graduates serve various educator roles in 96 North Carolina Counties with a concentration in districts close to UNC-CH and large, metropolitan districts.
Each of the advanced programs has an annual program review cycle during which its program data is reviewed and analyzed to inform program improvement activities. The employment data provide a critical evidence base for this work and are examined holistically with other quality assurance measures, such as exit surveys and ESAs.
We are also monitoring how our candidates move into positions outside of NC Public Schools, such as agency, institutional, or higher education settings, and move to other states. Our unit is developing these data sources to complement NCDPI data to provide a richer and more accurate reflection of completer employment.
Graduates employed by NC Public Schools in Prepared Licensure Areas
Data sources: NCDPI Employment data May 2021 and UNC SOE Degree Completion Data
Data Source: NCDPI Employment Data
Note: This map includes UNC graduates who serve in teaching, student service or administrative roles in an NC Public School as of May 2021. The data may include BA, MAT (Master of Arts in Teaching), MEDX, MSA, MED School Counseling and other degree completers.