The ABR Oral Exam

Introduction

After 40 years of administering the medical physics oral exam in Louisville, Kentucky, the ABR plans to move the exam to Tucson, Arizona, in 2019. (The 2018 oral exam will remain in Louisville.) This transition provides a good opportunity to review the nature and purpose of the oral exam.

Also called Part 3, the oral exam is the final one in a sequence of three exams that together form the basis for ABR certification in medical physics. Material on the oral exam is based on the knowledge and skills that a competent medical physicist should have upon completion of a medical physics residency. This material is essentially the same as that on the Part 2 computer-based exam, but with a stronger emphasis on clinical medical physics, clinical judgment, and communication. Successful completion of Parts 2 and 3 demonstrates the level of achievement necessary for a medical physicist to practice independently. The content guide for the oral exam is available on the ABR website.

The Structure of the Exam

The questions on the oral exam are in five categories, given by five examiners in five 30-minute sessions. Each examiner asks one question from each category, and the order is varied to ensure equal coverage of each category. This schedule is shown in the table below.

Examiners	Question Order
Examiner 1	Q1	Q2	Q3	Q4	Q5
Examiner 2	Q2	Q3	Q4	Q5	Q1
Examiner 3	Q3	Q4	Q5	Q1	Q2
Examiner 4	Q4	Q5	Q1	Q2	Q3
Examiner 5	Q5	Q1	Q2	Q3	Q4

All ABR exam scoring is criterion referenced, meaning that a passing standard is established in advance of the exam. The exam is not graded on a curve, and there is no set percentage of failures.

Ensuring Fairness

One of the concerns candidates have about an oral exam is the possibility of subjective grading. Another concern is expressed as, "What if I don't hit it off with an examiner?" We will address both concerns below.

The first step in producing a fair exam is to establish an equitable and relevant method for developing each question. A committee of volunteer medical physicists who have recognized expertise in one of the three medical physics specialties develops the questions. These committees also critically analyze the questions to ensure they are clear, unambiguous, and do not have a regional bias. Committee members develop answers to guide the examiners, suggest follow-up questions, and select the questions for each Part 3 exam. A content grid for each of the three medical physics specialties ensures each exam is properly balanced. For diagnostic medical physics, the content grid also establishes the correct balance of MRI and ultrasound for each Part 3 exam.

When the examiners gather before the exam, all who will ask each question meet to discuss the it, including any nuances of which they should be aware. After a question is given as part of an exam, we analyze how it performed by asking, "What is the average score for each question?" We also analyze the discrimination, a statistic that compares the performance of the top scorers with the performance of the bottom scorers for each question. Clearly, we expect the better overall performers to receive higher scores on each question.

Examiners also receive extensive training before the administration of the exam. Topics include:

• putting candidates at ease,
• supporting candidates who are struggling with a question,
• scoring questions,
• managing time during the exam administration, and
• many others.

New examiners are given extra training and must observe experienced examiners before they are allowed to examine.

To ensure fairness, we randomly assign candidates to the available time slots; thus, first-time takers and returning candidates are grouped together. The only information the examiners and panels have about a candidate is his or her name. We also take specific actions to ensure there is no significant personal or professional relationship between an examiner and a candidate.

Several times during the exam, the ABR trustees and governor conduct a formal observation of each examiner to be sure they are examining as we expect. Finally, to reduce examiner fatigue, no examiner sees more than three candidates in a row.

The examiners are grouped in panels of five. Following each exam session, the panel meets to review the session and discuss each candidate's performance. Because each examiner asks one question from each category, the effect of any one examiner on performance results in any one category is minimized.

Statistics

The ABR keeps statistics for each question, as well as statistics for each examiner. We track the average score each examiner assigns to candidates and the percentage of high scores and low scores. We also keep statistics for each panel, which demonstrate that the histogram of panel performance is quite narrow. The variation in performance from panel to panel is very small.

As mentioned by Dr. Kanal in the last newsletter, there is no statistical difference between the exam results of female and male candidates. The scores from recent years are shown below:

First-Time Takers
Exam Dates	Average Percentage Conditioning	Average Percentage Failing	Average Percentage Passing	Average Total Examinees
2014-2016	13%	20%	67%	230

A failure in one of the five categories on the Part 3 oral exam results in "conditioning" the exam, and the candidate must retake the failed category.

We are always happy to receive comments about the exam. If you wish to provide comments or have questions, we encourage you to contact us at any time.