About CAEP Annual Reporting

As part of CAEP accreditation requirements, UNC-CH shares data demonstrating the effectiveness of its completers’ instruction on P-12 student learning and development, and completer and employer satisfaction with the relevance and effectiveness of preparation.

Measure 1: Impact on P-12 Learning and Development

Measure 1: Indicators of Teaching Effectiveness

Measure 2: Satisfaction of Employers – Initial Programs

Measure 2: Satisfaction of Employers—Advanced Programs

Measure 3: Candidate Competency at Completion—Initial Programs

Measure 3: Candidate Competency at Completion—Advanced Programs

Measure 4: Ability of Completers to be hired

Measure 1: Impact on P-12 Learning and Development

These student growth data are in alignment with CAEP Annual Report Measure 1 (Initial): Completer effectiveness. (R4.1) and demonstrate completer impact in contributing to P-12 student-learning growth.

caep_chart1

The above graph shows P-12 student growth collected annually through North Carolina Educator Value-Added Assessment System (EVAAS) for beginning teachers (3 or less years of teaching) prepared by UNC-CH. The bars show the percentage of UNC-CH beginning teachers who do not meet, meet, or exceed expected growth for and all beginning teachers in North Carolina public schools.

2017-2021 EVAAS data shows that 80% or more UNC-CH completers meet or exceed growth expectations annually. While the percentage of ratings may vary across years, UNC-CH completer generally outperform beginning teachers across the state on student growth prior to the COVID-19 Pandemic.  In 2018-19, 42% of UNC-CH prepared beginning teachers exceeded growth expectations comparing to 13% at the state level. The trend was disrupted by COVID-19. NCDPI did not collect P-12 student assessment data for 2019-20. 2020-21 EVAAS data show that 84% of UNC-CH prepared beginning teachers meet or exceed growth expectations, but the distribution of ratings is different compared to previous years, which is likely caused by the COVID-19 pandemic impact on P-12 student learning.

UNC-CH monitors EVAAS data annually seeking opportunities make targeted improvement in program areas and their coursework. We also monitor these data as they are a key component in the new accountability model for North Carolina EPPs. Our use of these data for program improvement post-COVID-19 will differ from their use prior to the pandemic. Of note, we are eager to ascertain the if performance trends for UNC-CH and the state data return to pre-pandemic rates—where UNC-CH has higher rates of teachers meeting or exceeding student growth estimates that the state average—or if the trend will shift.

Data Source: NCDPI EPP Performance Report

Back to Top

Measure 1: Indicators of Teaching Effectiveness

These teacher effectiveness data are in alignment with CAEP Annual Report Measure 1 (Initial): Completer effectiveness. (R4.1) and demonstrates completer effectiveness in applying professional knowledge, skills, and dispositions.

This set of data/graphs shows school administrators’ ratings of UNC-CH prepared beginning teachers (3 or less years of teaching) on the NCEES criteria (North Carolina Educator Evaluation Systems):

  • Standard 1: Teachers Demonstrate Leadership
  • Standard 2: Teachers Establish a Respectful Environment for a Diverse Population of Students
  • Standard 3: Teachers Know the Content They Teach
  • Standard 4: Teachers Facilitate Learning for Their Students
  • Standard 5: Teachers Reflect on Their Practice

NCDPI collects teacher effectiveness data on beginning teachers each year except for Covid impacted 2019-20.  School Administrators rate at the level at which beginning teachers meet Standards 1-5 as they move from ratings of ‘Developing’ to ‘Distinguished’. Effective 2020-21, at the end of their third year beginning teachers must be rated ‘Proficient’ on Standards 1-5 on the most recent ‘Teacher Summary Rating Form’ in order to be eligible for the Standard Professional 2 license. New teachers are more likely to be rated lower on the evaluation standards as they are still learning and developing new skills and knowledge.

2016-17 through 2018-19 and 2020-21 evaluation results indicate that UNC-CH prepared beginning teachers consistently receive high ratings on all NCEES criteria. 98% to 100% of UNC-CH beginning teachers are rated proficient, accomplished, or Distinguished on all NCEES criteria, which is consistently higher than statewide ratings. Specifically, as shown in the radar graph for 2020-21, 100% of principals rated our beginning teacher as more or much more effective in all surveyed teaching practices.

Before the pandemic, a comparison between 2016-17 through 2018-19 teacher evaluation data finds that UNC-CH beginning teachers improved effectiveness over all teacher standards. We consider this an expected result of continuous improvement of our programs. Due to covid-19, NCDPI did not administer the survey 2019-20. The data shows that UNC-CH prepared beginning teaching continue to be rated as 100% proficient or above on all teaching standards.

The program used these data as an important marker of principal/employer feedback on our completers. We triangulate these data with those from the NC Employer survey to generate a more holistic view of how graduates are perceived by their supervising administrators during their first years of teaching. We see some points of alignment between the data sources and some areas for further investigation. One area for further analysis and action related to Standard 3: Teachers know the content they teach. Principal ratings and the Employer survey differ in their ratings. As a result, the program is actively considering how it examines content knowledge expertise at admission and as developed throughout the program.

Data Source: NCDPI EPP Report Cards

caep_chart2
caep_chart3
caep_chart4
caep_chart5
caep_chart6
caep_chart7

Back to Top

Measure 2: Satisfaction of Employers – Initial Programs

NC School Employer survey data are in alignment with CAEP Annual Report Measure 2 (Initial): Satisfaction of employer and stakeholder involvement. (R4.2|R5.3)

First administered in 2016-17 by NCDPI and the Education Policy Initiative at Carolina (EPIC), the NC School Employer survey asks school principals or assistant principals to rate the effectiveness of first-year teachers on 29 teaching tasks that are aligned with the state’s professional teaching standards, comparing to other NC prepared, out-of-state, and alternative entry first-year teachers. The first-year teachers are rated on a scale of 1 to 5: Much Less Effective, Less Effective, Comparable, More Effective, and Much More Effective.

This set of graphs shows principals’ overall satisfaction with UNC-CH first year-teachers, and mean rating scores for each of the five professional standards comparing to other EPP groups.

NC School Employer survey results from 2016-17 to 2020-21 indicate that UNC-CH graduates are highly effective in teaching, and principals are generally satisfied with our students.  Over the past five year, 92% or more of principals reported that UNC-CH candidates performed different teaching tasks comparable, more effectively, or much more effectively than first-year teacher prepared by other programs and states. The percentage of employer responses indicating satisfaction increased from 92% in 2016-17 to 98% in 2020-21, which is considered an indication of continuous improvement of UNC-CH program and candidate quality.

UNC-CH prepared beginning teachers consistently receive comparable or higher ratings than other EPPs or States on all NC professional teaching standards over years. The 2020-21 radar graph, in particular, demonstrates that UNC-CH graduates received higher ratings than other teach preparation programs on all five teacher standards.

The program used these data as an important marker of principal/employer feedback on our completers. We triangulate these data with those from the NC Principal evaluations to generate a more holistic view of how graduates are perceived by their supervising administrators during their first years of teaching. We see some points of alignment between the data sources and some areas for further investigation. As noted previously, one area for further analysis and action related to Standard 3: Teachers know the content they teach. Principal ratings and the Employer survey differ in their ratings. The Employer Survey rates UNC completers lowest on Standard 3, yet still higher than other non-UNC prepared teachers. UNC-CH is examining different ways to address and assess content knowledge development in our program and how it is demonstrated in the field.

Data Source: NCDPI – NC Employer Survey

caep_chart8
caep_chart9
caep_chart10
caep_chart11
caep_chart12
caep_chart13
caep_chart14

Back to Top

Measure 2: Satisfaction of Employers—Advanced Programs

These data are aligned with CAEP Annual Report Measure 2 (Advanced): Satisfaction of employer and stakeholder involvement. (RA4.1|RA5.3)

To support EPPs’ continuous improvement efforts and accreditation needs, NCDPI started providing employment data for EPP completers in 2018 and set to release this data annually in April. NCDPI employment data identifies all UNC SOE completers who have graduated with a degree and been employed as an educator in North Carolina public or charter schools. These data includes completers’ identifications, employee information, completers’ job title, and the degree levels earned at UNC.

Since UNC-CH is using proxy measures to address Employer Satisfaction with our advanced candidates, the unit is actively working to develop and implement stronger feedback mechanisms for its graduates. One action is the planning and development of an SOE employment database with the 3 years of NCDPI employment data, and plan to expand this database yearly when new data become available. NCDPI Employment data has been connected with SOE completion data to add corresponding completion data: completion term, degree completed, academic plan and subplan, Gender, Ethnicity etc. With this data, we are able to track employment by program and cohort, map employment distribution in North Carolina. As this database continues to grow with more years of employment data, we will be able to track our completers’ career transition and job retention, and employer satisfaction.

Highest Employing NC Public School Units of UNC Completers 2020-21

caep_table1

Data Source: NCDPI 2020-21 EPP Performance Report

Back to Top

Measure 3: Candidate Competency at Completion—Initial Programs

This set of assessments align with CAEP Annual Report Measure 3 (Initial): candidate competency at completion. (R3.3) and shows UNC-CH candidates are meeting program expectations and ready to be recommended for licensure.

At program completion, UNC-CH initial teacher candidates are evaluated on multiple performance metrics which include NC Licensure Exams, edTPA, NC LEA/IHE Certification of Teaching Capacity and Course Grades to determine qualification for program completion and licensure recommendation.

The Federal Tile II reporting collects Licensure assessment data including Praxis II and edTPA for all licensure areas, Foundations of Reading for Elementary Education, and calculate pass rate annually to demonstrate EPP quality. UNC-CH pass rate has been consistently 95% or above and significantly higher than North Carolina statewide average.

NCDPI monitors licensure Exam pass rates for all NC Educator Preparation Programs through NC Educator Preparation Program Dashboard. UNC-CH pass rates are generally 20% higher than other NC EPPS as a whole or by licensure areas.

For the past three years, UNC-CH candidates received an average edTPA score of 45 or above, which is 8 points above the passing score and within this same domain, UNC-CH candidates’ edTPA scores were higher than candidates from North Carolina statewide and the national average.

caep_chart15

Data Source: Title II Reports

caep_chart16

Data source: NC DPI Educator Preparation Program Dashboard

caep_chart17

Back to Top

Measure 3: Candidate Competency at Completion—Advanced Programs

This set of assessments align with CAEP Annual Report Measure 3 (Advanced): candidate competency at completion. (RA3.4), and shows candidates are meeting program expectations and ready to be recommended for licensure.

Since North Carolina does not required licensure exam for advanced licensure areas, UNC-CH advanced programs (MEdX-ECIFS, MSA and EDD Education Leadership) evaluate students competency through course embedded assessments (ESAs) and collects ESA data and other program artifacts in our e-Portfolio platform, Taskstream. All ESA rubrics are aligned with corresponding national standards.

The preliminary ESA data shows that UNC-CH are scored 90 or more on most of national stands, with a variation from 87 to 100 among standards. These data provide evidence of candidate competency and an opportunity to identify area/standards for improvement.

Each of the advanced programs has annual program review cycle during which its program data is reviewed and analyzed to inform program improvement activities. The data generated by the ESA forms a critical evidence base for this work and are examined holistically with other quality assurance measures, such as exit surveys and employment data.

EDD Ed Leadership: NELP District Level Standards Assessed in EdD Coursework (Fall 2020)

caep_table2

MEdX

All data presented in this section align to Early Intervention/Early Childhood Special Education (EI/ECSE) standards. At present MEDX-ECIFS Embedded Signature Assessments (ESAs) assess all of the seven EI/ECSE standards; ESA data are presented in data tables below.

caep_table3
caep_table4

Data Source: Course embedded assessment data collected in Taskstream

Back to Top

Measure 4: Ability of Completers to be hired

This set of data aligns with CAEP Annual Report Measure 4 (Initial and Advanced): Ability of completers to be hired in positions for which they have prepared.

The ability of initially licensed teachers to be hired is best reported through the licensure and employment data from NCDPI EPP Performance Dashboard – Initial Employment and Retention .

The ability of advanced program completers to be hired into positions aligned with their programs of study utilizes employment data from NCDPI. For advanced completers we track employment in NC public schools and, more specifically, employment in certain job roles. The exception are graduates from UNC-CH’s MEDX ECIFS program. Many candidates return to the agency or private settings upon graduation and, therefore, are not present in NCDPI employment data.

The employment table shows that 60% of 2019-20 of MSA and MEdX ECIFS programs are employed by North Carolina Public Schools in their prepared licensure areas one year after program completion in 2020-21. Half of EDD Education leadership graduate served in principal or assistant principal role which is considered on the route to senior leadership. These data do not track graduates who works in private schools, higher education institutions, or other sector of educational institutions.

As demonstrated by the employment distribution maps, UNC-CH graduates serves various educator roles in 96 North Carolina Counties with a concentration on Wake, Durham, Orange and Chatham school districts.

Each of the advanced programs has annual program review cycle during which its program data is reviewed and analyzed to inform program improvement activities. The employment data provide a critical evidence base for this work and are examined holistically with other quality assurance measures, such as exit surveys and ESAs. We are also monitoring how our candidates move into positions outside of NC Public Schools, such as agency, institutional, or higher education settings, and move to other states. Our unit is developing these data sources to complement NCDPI data to provide a richer and more accurate reflection of completer employment.

Graduates employed by NC Public Schools in Prepared Licensure Areas

caep_table5

Data sources: NCDPI Employment data May 2021 and UNC SOE Degree Completion Data

 

caep_chart18

Data Source: NCDPI Employment Data

Note: This map includes UNC graduates who serves in teaching, student service or administrative roles in an NC Public School as of May 2021. The data may include BA, MAT, MEDX, MSA, MED School Counseling and other degree completers.

Back to Top