Fig. 1 Nearly 75 percent of Diplomates rated the ABOS WLA as “very good’ or “excellent.”
Courtesy of The American Board of Orthopaedic Surgery


Published 11/30/2019
John (Jack) M. Flynn, MD, FAAOS

ABOS WLA: 2019 Survey Results and Changes to 2020 Program

In January, the American Board of Orthopaedic Surgery (ABOS) posted 101 Knowledge Sources to the ABOS website, launching the inaugural ABOS Web-Based Longitudinal Assessment Program (ABOS WLA). Nearly 10,000 ABOS Diplomates participated—about 55 percent of those who were eligible—as a way to meet their assessment requirements for ABOS Maintenance of Certification (MOC).

The ABOS WLA is based on adult-learning principles (assessment for learning) and was launched as a pilot program to give ABOS Diplomates an alternative to the standard oral and computer-based assessment options that have been offered for many years. The Board thought it was important to allow all Diplomates the chance to participate in the first-year pilot program and is pleased that so many decided to try it. The only Diplomates not eligible in 2019 were those who had recertified early and had not yet entered their new 10-year cycle.

Participating Diplomates chose 15 Knowledge Sources for in-depth review. During the assessment period, they completed two questions per knowledge source. Nearly 300,000 questions were administered during the five-week assessment window. More than 98 percent of Diplomates who participated earned a Quality Year—successfully answering at least 24 of the 30 questions.

Diplomates were able to answer the questions on their own computers at a time and pace convenient for them during the five-week window. Some chose to answer all 30 questions at once, whereas others decided to complete the process over several sessions.

Once the assessment period concluded at the end of May, the Board engaged RTI International, a research institute, to survey Diplomates about their experiences with the ABOS WLA. The survey included eligible Diplomates, including those who took the ABOS WLA and those who chose not to. The ABOS would like to extend a thank you to the nearly 8,000 Diplomates who completed the survey. Much of the feedback was positive, and the ABOS received many worthwhile suggestions. This article presents some highlights from the survey.

Survey outcomes

The ABOS aimed to ensure that a true representation of practicing orthopaedic surgeons responded—it appears to have been successful. The survey collected the highest number of responses from those practicing orthopaedic sports medicine, followed by adult reconstruction and general orthopaedics. A plurality of respondents reported being in group private practice, but many respondents came from multispecialty groups, hospitals, or solo private practices.

Nearly 75 percent of Diplomates rated the ABOS WLA platform as “very good” or “excellent,” (Fig. 1) and more than 96 percent indicated that they were “very likely” or “extremely likely” to continue to participate next year (Fig. 2). Respondents gave very high marks to the ABOS WLA platform and the Knowledge Sources that were available (Fig. 3). Most respondents also indicated that the ABOS WLA helped improve their practice. Only a few Diplomates reported having technical difficulties with the platform or needing to contact the ABOS for support; however, when they did contact the ABOS, most were satisfied with the help received. The survey found that the vast majority of those who participated in the ABOS WLA had a positive experience and want to continue with it in the future.

Fig. 1 Nearly 75 percent of Diplomates rated the ABOS WLA as “very good’ or “excellent.”
Courtesy of The American Board of Orthopaedic Surgery
Fig. 2 More than 96 percent indicated that they were “very likely” or “extremely likely” to continue to participate next year.
Courtesy of The American Board of Orthopaedic Surgery
Fig. 3 Diplomates were asked how strongly they agree or disagree to several statements about the ABOS WLA Program.
Courtesy of The American Board of Orthopaedic Surgery

ABOS staff read all of the open-ended responses and appreciate the Diplomates taking the time to provide valuable, candid feedback. Via the open-ended questions, Diplomates indicated that they would like more time to answer each question and would like to know in advance from which Knowledge Source each question was derived.

The ABOS also was happy to hear from many Diplomates who were eligible but chose not to participate. The top reason was that they wanted to take one examination a decade, which is still a possibility. The ABOS offers nine different Practice-Profiled Recertification Examinations that contain only subspecialty-specific questions (no general orthopaedic questions). The ABOS Oral Recertification Examination is also an option. Other reasons Diplomates did not participate in the ABOS WLA included the five-week assessment window not being convenient or respondents having highly subspecialized, very specific practice types.

The Board was happy that an overwhelming majority of Diplomates considered the ABOS WLA a successful assessment option. In response to the feedback, the ABOS will make several improvements for 2020. Diplomates will know in advance from which Knowledge Source each question is derived. There will be new Knowledge Sources and questions for each subspecialty, and many of the top 2019 Knowledge Sources will be available again; if a Diplomate did not choose a Knowledge Source in 2019, he or she will have another opportunity in 2020. Diplomates can see all of the 2020 Knowledge Sources starting Jan. 2, 2020, by logging in to the ABOS Dashboard. The ABOS WLA Assessment Window will run from April 13, 2020, to May 18, 2020.

The 2020 ABOS WLA is open to all Diplomates who have a Certification expiration year of 2026–2029 and to Diplomates with a Certification expiration of 2019–2025 who earned a Quality Year in 2019. The ABOS WLA meets the assessment portion of MOC. Those who participate in the ABOS WLA also must meet the other MOC requirements, including submitting an application and application fee, submitting a case list, being peer reviewed, and participating in continuing medical education and self-assessment examination activities. More information on the MOC requirements can be found at

More information about the ABOS WLA can be found in the Diplomates section at Call the ABOS offices at 919-929-7103 with any questions about this new program.

In addition, the ABOS communicates with Diplomates primarily through email; ensure yours is correct by visiting, logging in, and confirming that it is accurate.

View more survey information at

John (Jack) M. Flynn, MD, FAAOS, is chair of the ABOS MOC Committee.