Assessment in Medical School

Here is a guest blog post from my good friend Dr. Brad Bahler. The problems revolving around our current assessment techniques are not unique to K-12. Post secondary medical schools are also searching to broaden their definition of excellence.

By Dr. Brad Bahler

I was recently reading an interesting book called “Bloodletting and Miraculous Cures” by Vincent Lam. Part of the novel follows a group of students through application to medical school. I was brought back to my own medical school application days. I admit that it always seemed a bit of a chess game to position yourself to “get into medicine”. You would of course need top grades, you would need a good MCAT score (which is a medical school admissions test that at one point was very extensively used), and you would need to do well in a 20 minute “impress the admissions members” interview. In the novel 2 students are working together to study through university courses. “Ming” is a hard studying top-level achiever, and “Fitzgerald” is a man who becomes engrossed in concepts, engulfed in learning at the expense of top-level marks. Eventually Fitzgerald helps Ming understand material she is having trouble with, and she is accepted to medical school while he is not. In the below paragraph she has just reviewed one of his exam papers on which she outscored him easily despite his seemingly superior understanding of the concepts;

“ . . . you understand this stuff. You’re losing marks on detail. The Krebs cycle – you know it better than I do. The problem is the way you study and write. She said this not only to be kind, but because she found his answers elegant and insightful. Ming’s own responses were always factually complete; in point form, convenient to check off for a perfect score. Fitzgerald seems to disregard the assigned value of questions, and in some three–inch space he cramped his writing into tiny letters in order to include the essay-length breadth he felt appropriate. In another section where a page was allotted, he wrote four lines and drew a diagram that, to him, encapsulated the entire issue.” Page 15

Later in the book, Fitzgerald becomes determined to gain entry to medical school, and to do so he completely changes his studying habits in order to attain those needed top marks;

“ . . . Fitzgerald wondered whether his biology and biochemistry lectures were no longer real – perhaps they were only the means to an end. He had previously enjoyed the ideas and concepts but now, even as he became more obsessive about the details and patterns of facts, he hated knowing that his marks were soaring . . . “ Page 60

With his newfound tactics he is eventually accepted, but so obviously his love of learning killed in the process.  This work of fiction unfortunately struck very close to home as I look at those I went to medical school with. It raises the question, is this really what it is like; and if so, does this really select the doctors we want for the future? Because while I accept that you absolutely cannot finish medical school without the academic stamina, core knowledge and mental capabilities necessary to understand and apply information, I refuse to accept that the thing that predicts success in medical school is GPA or MCAT score.  And by the way, this comes from someone who had a top GPA and MCAT score.

Now medical school admissions have come a long way since I applied around 13 years ago. Of that there is no doubt. A quick glance at the University of Alberta web site will show you that 50% of the admissions criteria is GPA and MCAT based and 50% is personal activities, interview marks and references. This is a slow but sure swing away from test scores being the dominant factor. But I am not sure any university has yet to hit the nail on the head when it comes to the best admissions formula.

To me the big question is, what do I want in a doctor? Clinical competence has to be part of the picture of that there is no doubt. Simply put, regardless of if you ask practicing physicians, medical residents, medical students or patients, competence comes out at or near the top of everyone’s “what’s important” list. But I also value compassion, communication skills, patience, the ability to teach, and the ability to continue to learn and adapt, the ability to endure loss, endure stress and look at the global picture of prevention and community need. I am not sure any of our current undergraduate assessment methods capture those characteristics. I also will argue that gauging someone’s future ability to achieve clinical competence has very little to do with what a GPA or MCAT tells you. I also think it has very little to do with your written test marks in medical school, which is likely the reason why most medical schools are pass/fail or qualitative. Clinical competence is nebulous, it is elusive, it is difficult to define and when you see it in someone it is amazing. It is not regurgitation of fact; it is constant adaptation and integration of a number of disease and patient factors. In short it is adaptive expertise. I won’t bore anyone with a 3 page long explanation of why this is what I think medical students should strive for. For anyone that is interested I would check out the work of Maria Mylopoulos and Glenn Regehr such as their article in Medical Education, Volume 43, Issue 2, pages 127-132, February 2009.

So is anyone close to routine identification of that special group of people that have potential to achieve true clinical competence as well as that laundry list of other very special traits? If you look at medical schools around Canada you will see change. The MCAT is on it’s way out in many places. Work is being done on innovative ways to improve selection. There is a very recent article that should still be publically available at CMAJ;

http://www.cmaj.ca/earlyreleases/20oct10-the-changing-dynamic-of-medical-school-admissions.dtl

I highly suggest anyone who has an interest in changes to the admission processes across the country read it. I also think this is reflective of the global movement in education at every level. As a society we are starting to see that marks are not everything, and in fact, they may be in many circumstances the antithesis of everything that actually matters. It is time for further change and the process will be slow and painful for many well-established programs. But if we are asking our students to become adaptive experts then as Universities we need to become adaptive experts when it comes to selection and find that special group of people to lead health care in the century to come. I don’t claim to have all the answers, but I am willing to learn, adapt and foster change at any level I can. Because when I am nursing home age, I want a doctor that can talk to me, understand me as a person, have the capabilities to manage my illness, make me feel safe and comfortable with what I have left of life, and perhaps once and a while crack a dirty joke to lighten the mood.

DR. BRAD BAHLER

*Please note any opinion in this article is not representative of any University the author may have affiliation with.