Skip to main content

Assessing the Competency of EMS Educators

Assessing the Competency of EMS Educators

The Results

This study was designed to measure the ability of EMS personnel credentialed or licensed by their state EMS governing boards as instructors to assess skill assessment competencies in a videotaped simulated NREMT psychomotor examination.  Participants in both the calibration phase and the execution phase were sought from a national pool of credentialed EMS educators. 

Prior to the delivery of the final survey instrument to the respondents, the instrument was field tested for reliability as well as calibrated for accuracy in the following manner. A video of a simulated emergency scenario and a student assessment were distributed to 20 EMS educators. Each of these participants possessed a bachelor’s degree or higher.  The teacher-observers were asked to view the video and assess the entry-level competency of the portrayal of specific EMS skills of each of the actors.  At least 10 days but no more than 15 days after the initial deadline for completion, the same group was asked to view the film again and to score the same actors on their display of entry-level competency. The respondents in this field test were identified by name so as to be able to reach a consensus should there be initial disagreement among the teacher-observers.   

EMS educators from around the country, who are credentialed or licensed to teach at the EMT level or higher, were the pool for this study.  These participants were solicited to participate via email and professional listserv pages.  The primary listserv that was used was that of the National Association of EMS Educators (NAEMSE).  These educators were emailed a letter outlining the need for their participation in this project and noted the data would be shared with the respondents upon completion of the study.  This email listed a website link and log-in information to access the video and the accompanying survey. 

The researcher-designed survey was administered via Survey Monkey, an internet-based survey.  All questions in the survey instrument were closed-ended so as to facilitate quantitative results.  No survey was found that addresses the data solicited by this study. Consequently, all questions were researched, developed, and composed by the researcher.  All questions were based upon the psychomotor skills examinations sheets from the NREMT. 

Data Analysis

The questionnaire employed a Likert scale ranging from one to five, with three being neutral. In addition, a limited number of open-ended questions were employed to seek clarification as to why an EMS educator might feel that the student did not display entry-level competency.  Only positive responses were included in the survey. The survey questions were analyzed within their specific cluster as either competent or not competent

Findings of the Study

The respondents provided their professional opinions as to whether the student being evaluated had displayed Entry-Level Competency (ELC) as a paramedic.  Their response was based on a yes or no answer. Eighty-eight percent of the respondents reported that the student did not display ELC, while 17 percent reported that the student did exhibit competency. This distribution is represented in Figure 1.

In this sample of 95 EMS educators, 84 of them (88 percent) identified the student as not having entry-level competency. Using 80 percent as a benchmark for ability to identify entry-level competency, a z-test determined that 88.4 percent was significantly greater than the benchmark (p = .040). 

In the event the student was evaluated as not competent, the respondents indicated that the student lacked both theoretical background as well as the practical, empirical knowledge and expertise in the provision of prehospital emergency medical care. Since the survey question regarding an explanation as to why the student was not competent was open-ended, the respondents provided reasons as to why they considered the student not minimally competent. These reasons range from lack of knowledge in EMS issues, poor leadership skills, to ineffective assessment of the patients or the situation, and are reported in Appendix G.  This table reveals that the majority of the respondents cited lack of knowledge among the entry-level students as compared to that of a minimally competent paramedic. This was indicated by poor leadership, on the part of the student, or little to no control over the emergency scene safety. Ineffective communication among the team members was cited the least among the EMS educators.  

Practical Significance

Eighty-three percent of the respondents replied that the paramedic students being evaluated had displayed Entry-Level Competencies (ELC)while 17 percent of the respondents did not. This is an indication that the majority of the targeted respondents agreed that paramedic students had displayed or could attain the Entry-Level Competencies within the expected time.

Most past studies have explored the entry-level competencies (Edwards, 2011;Whyte,Madigan & Drinkwater, 2011; Southgate et al. 2001; Holmboe, 2004). The thoughts and perspectives of entry-level competencies have not been explored. Therefore, a quantitative research design was employed in this study to investigate whether EMS educators are proficient in the identification of the competency of entry-level Emergency Medical Technician students.

Drawing on the significant writing from theoretical and empirical perspectives, this section presents an integrated account of the study’s findings and a discussion. First, the majority of the EMS educators can identify competent paramedic students who perform skills as outlined by the NREMT psychomotor examination. Secondly, there is a weak correlation between those educators who can identify competency versus non-competency and the degree of their acceptance and implementation of the National EMS Education Guidelines and Standards. This was supported by the fact that the study’s findings found a weak relationship between the two variables.

Finally, there is a weak correlation between those educators who can identify competency versus non-competency and the level of their formal education. The findings showed a weak relationship between adequate competencies as compared to their level of formal education. This weak relationship has been supported by the research of Kevorkian, Rintala, and Hart (2001) and Watson et al. (2012).

About the author:

Bill Young began his EMS career in 1975 with a small fire department near Williamsburg, KY. They began running first responder calls long before the phrase ever existed. In addition to Kentucky, his career has taken him to Tennessee, Colorado, Georgia, and Kansas. He has served as a street medic, training officer, supervisor, state regulator, and educator. He recently completed his doctoral studies in education at the University of the Cumberlands. Currently, he is an Assistant Professor and the Program Director at Eastern Kentucky University in the Fire and Paramedicine Science Department.



Edwards, D. (2011). Paramedic preceptor: work readiness in graduate paramedics. The clinical teacher, 8(2), 79-82.


Holmboe, E. S. (2004). Faculty and the observation of trainees’ clinical skills: Problems and opportunities. Academic Medicine, 79(1), 16-22.


Kevorkian, G., Rintala, D., & Hart, K. A. (2001). Evaluation and promotion of the clinician-educator: The faculty viewpoint. American Journal of Physical Medicine & Rehabilitation,80(1), 47–55.


Southgate, L., Campbell, M., Cox, J., Foulkes, J., Jolly, B., McCrorie, P., et al. (2001) The General Medical Council’s performance procedures: The development and implementation of tests of competence with examples from general practice. Medical Education, 35, 20–28.

Warrren, S. (2012). Volume versus outcome: More emergency medical services personnel on-scene and increased survival after out-of-hospital cardiac arrest.  Resuscitation, 4(5), 22-36.

Whyte, D. G., Madigan, V., & Drinkwater, E. J. (2011). Predictors of academic performance of nursing and paramedic students in first year bioscience. Nurse education today, 31(8), 849-854.

Published on March 07, 2017

Open /*deleted href=#openmobile*/