ABR NEWSGeoffrey Ibbott, PhD, ABR Board of Governors, Jerry Allison, PhD, J. Anthony Seibert, PhD, and
Matthew B. Podgorsak, PhD, ABR Trustees

Online Longitudinal Assessment (OLA) – A Step Forward in the Assessment of Cognitive Expertise

Why MOC and Why ABR OLA?

ABR OLA is an important addition to the ABR Maintenance of Certification (MOC) program. MOC programs exist to encourage all of us to remain current in our various specialties of medical physics. Because of the pressure of everyday work, we may neglect to maintain our knowledge of the details of our practices. This is especially true of things we don't do on a regular basis. We may also fail to keep up with new techniques as they emerge, and, finally, we may not pay as much attention to quality and safety as we should. This behavior is not at all unusual as we are all human and are under time constraints in our work environments. The four parts of MOC are designed to help us overcome these very human limitations.

Since the American Board of Radiology began requiring MOC participation in 2002, the most problematic part of the process has been the assessment of knowledge, judgment, and skills (Part 3, formerly known as "cognitive expertise"). This initially began as a decennial exam administered at Pearson VUE test centers throughout the US, as well as in Europe and Asia. This method of evaluation was widely disliked for many reasons. The two most commonly cited reasons were the following:

• Stress associated with the exam and diplomates' concern that they would not do well
• Time spent away from clinical practice to prepare for the exam

The aggregate performance of medical physics (MP) diplomates has varied widely over the years, which probably has been associated with a small number of test takers. Last year, the pass rate was well above 90 percent. The clear value of an assessment of knowledge, judgment, and skills is that it encourages diplomates to review the overall body of medical physics knowledge and to learn new clinically important concepts. However, a major disadvantage of the decennial exam was that it leads to cramming for the exam. This method of learning tends to cause diplomates to quickly forget the material, rather than fully incorporating it into their clinical armamentarium. In designing ABR OLA, the ABR is striving to keep valuable parts of knowledge, judgment, and skills evaluation while reducing two large deficiencies (the 10-year cycle with associated cramming and fear of poor performance).

How Will Online Longitudinal Assessment (ABR OLA) Work?

OLA questions will be based on "walking-around knowledge," which is information that a typical clinical physicist will know without a requirement for study or research.

• After answering a question, diplomates will immediately see the answer and an explanation of the answer.
• All questions will come with readily available references.

In the course of a year, each diplomate will receive 104 question opportunities. Each diplomate must attempt 52 questions per year but may attempt more as desired. After opening a question, the diplomate will have one or three minutes to answer it. Diplomates may decline to answer up to 10 questions per year without penalty, allowing them to avoid clinical areas in which they do not frequently practice. Diplomates who answer a question incorrectly will be given a similar question within a defined period of time.

A threshold for performance will be established, and diplomates will be kept informed of where they stand relative to the threshold. After reaching a summative decision threshold of 200 questions, diplomates will be expected to meet the threshold at the time of their next annual review. If the threshold is not met, a diplomate will have until the next annual review to meet it.

The date of OLA roll-out for medical physics has not been established but may be in 2019. In the meantime, diplomates currently meeting the requirements of MOC do not have any obligation to take a 10-year exam. The ABR will give an MOC exam this year for those who need one to meet regulatory requirements or who are not meeting MOC requirements.

ABR-OLA questions, like all ABR questions, are written by volunteers. We have already trained about 45 volunteers—15 in each medical physics specialty—in how to write OLA questions. These volunteers are from all types of clinical practice, including private practice, private hospitals, and academic institutions. They are also a mixture of medical physicists with master's and doctoral degrees. So far, we have developed about 150 questions in each of the three specialties. These questions will be further reviewed by content editors, medical physics subject matter experts, medial physics committees, and assembly groups. We are on track to present the OLA content as we roll it out.

We feel strongly that the new OLA program will be more effective as an evaluation of knowledge, judgment, and skills and should also be an easier program for diplomates to manage. The ABR OLA program is still a work in progress. Please check the ABR website for the latest updates and look for additional information in the AAPM Newsletter.

Professionalism and Ethics Questions on ABR Initial Certification Exams

On a separate topic, the ABR wishes to announce that, beginning in 2018, there will be an increased emphasis on professionalism and ethics on the Medical Physics Part 1 (general), Part 2, and Part 3 (oral) exams. A content guide is available on the ABR website.

These topics are very similar to the material presented in the following publications:
AAPM TG 159, "Recommended Ethics Curriculum for Medical Physics Graduate and Residency Programs"
AAPM Report TG 249, "Essentials and Guidelines for Clinical Medical Physics Residency Training Programs"
ABR/ACR/RSNA/AAPM/ASTRO/ARR/ARS "Online Modules on Ethics and Professionalism,"