Residency Facts

Why do the USMLE Step tests hold so much weight in the resident selection process?

The USMLE Steps are designed to test basic competencies required to practice medicine. However, they are commonly also used by resident selection committees in the evaluation of candidates. One reason why the USMLE carries so much weight in resident selection is that it is one of the few standardized and objective means of comparing medical students. Letters of recommendation are by their nature subjective, and in general tend to be overwhelmingly positive. Medical school grades are limited by variable standards between different schools (particularly since so many institutions have adopted pass/fail systems).

The USMLE tests are also considered a valuable predictive tool. Although somewhat controversial, studies have shown at least some correlation between USMLE scores and clinical performance during medical school and residency. There also appears to be a correlation between performance on the USMLE tests and performance on board certification tests later in the resident's career. The highest correlations appear to be with students at the extremes, whereas small differences in the mid range of USMLE scores are perhaps less helpful in distinguishing medical students' aptitudes.

Most recognize, of course, that the USMLE has limitations when used for the purpose of comparing applicants. Representing performance on just one or two days of testing, it is an imperfect summary of a student's medical knowledge and clinical skills. Students unhappy with their USMLE scores should be motivated to shine in other aspects of their application.

Reference:

Bernes ES, Brooks CM, Erdmann JB. Use of the USMLE to select residents. Academic Medicine. 68(10), October 1993, 753-759.

Which Step is most important in my application to residency?

Three USMLE exams must be taken in order to gain licensure. While the tests are designed to assess basic competencies for licensing physicians, scores are often also used as a metric for evaluating students applying to residency. All students have taken Step I by the time of residency application and most, but not all, have taken Step II. Step III is usually taken during residency, so it is not typically a factor in the residency application process.

A survey was issued to program directors across the country as part of a 2009 study to determine what factors are most important in their decisions about residents. Of the USMLE tests, program directors placed the most weight on the USMLE Step I, which was ranked second in overall importance after grades during the third year required clinical clerkships. The next most important factors were: grades during the elective clerkship for the specialty to which the applicant is applying (when applicable; e.g., a radiology clerkship for a student applying in radiology) and number of honors grades. In comparison, preclinical grades were considered less important, perhaps due to variability in grading systems across different medical schools. In some ways, the USMLE Step I test is like a standardized means of comparing students' performance during the preclinical years of medical school.

Reference:

Green M, Jones P, Thomas, JX. Selection Criteria for Residency: Results of a National Program Directors Survey. Academic Medicine 84(3), March 2009, pp 362-367.

How do the program directors across different specialties weigh the results of USMLE Steps I and II when making decisions about residents?

A 2009 study examined how residency program directors across different specialties weigh a variety of factors, including the USMLE tests. Specialties were assigned a competitiveness rating based on the percentage of matched students who were from U.S. medical schools. This placed ENT, orthopedic surgery, and plastic surgery among the most competitive specialties with family medicine, internal medicine, and neurology among the least competitive. As might be expected, the average USMLE score increased in relation to specialty competitiveness.

In general, Step I scores carry more weight than Step II scores. Among specialties considered 'highly competitive,' Step I scores were among the most heavily weighted factors. (The exception was plastic surgery, where Step I score ranked sixth in importance.) In fields considered 'moderately competitive,' the importance of a student's USMLE Step I score ranged from being the first most important factor (urology) to the ninth most important factor (pediatrics). In specialties considered 'less competitive,' it ranged from third most important (pathology) to ninth most important (internal medicine).

Program directors from less competitive specialties ranked Step II CK and CS scores higher in importance than did those in more competitive specialties. This was particularly true for family medicine where Step II CS and CK were ranked first and second in importance, respectively. Other specialties with a particularly strong emphasis on Step II CK included anesthesiology, physical medicine and rehabilitation, internal medicine, and neurology. Among highly competitive specialties, Step II CK was given most weight in radiology where it ranked fifth and neurosurgery where it ranked third.

Reference

Green M, Jones P, Thomas, JX. Selection Criteria for Residency: Results of a National Program Directors Survey. Academic Medicine 84(3), March 2009, pp 362-367.

I went to an osteopathic medical school and took the COMLEX exam but am applying to an allopathic residency. How would my COMLEX score convert to a USMLE score?

Students who attend osteopathic medical schools must take the COMLEX exams during their second and fourth years of medical school, similar to the timing of the USMLE Steps I and II exams. Many of these students decide to apply for allopathic residency programs that are approved by the ACGME as opposed to osteopathic residency programs approved by the American Osteopathic Association. In fact, in the 2001-2002 academic year, more graduates of osteopathic medical schools were in allopathic than osteopathic residencies. However, allopathic residency program directors may not be comfortable with the COMLEX scoring system and therefore students from osteopathic medical schools may feel pressure to take the USMLE exam as well or face a possible selection bias.

A 2006 study examined this issue by comparing scores in a group of students who took both exams and devised a means by which USMLE scores could be predicted based on a student's COMLEX score. The study found scores on the two exams to be highly correlated. The formulas are as follows: USMLE Step 1 Score = 69.97 + (0.24 x COMLEX-USA Level I Score), USMLE Step 2 CS Score = 102.2 + (0.18 x COMLEX-USA Level II Score). Despite this correlation, the tests do have important differences. Most notably, the COMLEX has many questions on osteopathic theory and technique. Another limitation of this study is that all the students were from the same institution, making it unclear if this data is representative of osteopathic students in general. While this study does provide a potential means for converting a COMLEX score into an approximate USMLE score, it is not known whether program directors would actually use this conversion. Furthermore, it is not definitively known whether an osteopathic student's success rate in the allopathic match is increased by taking the USMLE in addition to the COMLEX.

Reference

Slocum PC, Louder JS. How to Predict USMLE scores from COMLEX-USA Scores: A Guide for Directors of ACGME-Accredited Residency Programs. JAOA 106(9), September 2006, 568-567.