AI's Role in Transforming Medical Education: HYPS Tutoring and Ethical Learning

Advancements in artificial intelligence (AI) are reshaping the landscape of medical education, with potential ramifications for students worldwide. A recent study conducted by Stanford researchers has uncovered intriguing insights into the capabilities of AI, specifically ChatGPT, which has demonstrated its prowess by outperforming first- and second-year medical students in addressing complex clinical care exam questions. This revelation challenges conventional methods of medical education and underscores the importance of ethical learning with guidance from established thinkers like our HYPS alum team.

AI's Surge in Medical Education

ChatGPT, a prominent large language model AI system, has garnered attention for its remarkable abilities. Trained on the vast expanse of internet content, these systems serve as virtual chatbots, enabling users to input text and receive human-like responses promptly. Recent research has shown ChatGPT's competence in handling multiple-choice questions on the United States Medical License Examination (USMLE), a critical milestone for aspiring doctors.

However, the Stanford researchers sought to explore uncharted territory. They examined how AI, particularly ChatGPT (in its latest iteration, GPT-4), handled more challenging open-ended questions. These questions closely mimic the clinical reasoning assessments administered to first- and second-year Stanford medical students. The questions entail deciphering intricate patient cases and formulating diagnoses—a task that demands intricate clinical reasoning skills.

In a groundbreaking article published in JAMA Internal Medicine, the research team discovered that, on average, ChatGPT surpassed human test-takers by more than four points in the case-report section of the exam.

Eric Strong, a clinical associate professor at Stanford School of Medicine and a co-author of the study, remarked, "We were very surprised at how well ChatGPT did on these kinds of free-response medical reasoning questions by exceeding the scores of the human test-takers."

Alicia DiGiammarino, the Practice of Medicine Year 2 Education manager at the School of Medicine and co-author, emphasized, "ChatGPT and other programs like it are changing how we teach and ultimately practice medicine."

AI as a Successful Student

This study builds upon prior research involving GPT-3.5, ChatGPT's predecessor. In both instances, the researchers crafted complex clinical reasoning cases, mirroring real-life patient scenarios, to evaluate AI's performance. Test-takers, both human and AI, faced questions necessitating paragraph-long answers.

Handling these questions was far more intricate than the multiple-choice questions typically encountered in the USMLE. While the latter mostly tests information recall, the former demands comprehensive open-ended responses—a far more challenging endeavor.

One challenge the researchers encountered was prompt engineering. ChatGPT draws knowledge from the entire internet, occasionally misinterpreting healthcare-centric terms. To mitigate this, questions were refined to ensure the AI comprehended medical terminology accurately. Afterward, ChatGPT's responses were compared with those of medical students who had tackled the same cases.

In the previous study with GPT-3.5, the AI's performance was described as "borderline passing." However, GPT-4 exhibited a significant improvement, scoring an average of 4.2 points higher than human students and achieving a passing rate of 93%, compared to the students' 85%.

Notably, while ChatGPT excelled, it was not without flaws. Confabulation, the inclusion of false details in responses, was a concern. However, this issue was notably reduced with GPT-4 compared to GPT-3.5.

Redefining Medical Education

The influence of AI on test-taking integrity and curricula design is already evident at Stanford School of Medicine. Exams have transitioned from open-book (with internet access) to closed-book, requiring students to rely on memory alone. Although this approach has its merits, it does raise concerns about students' ability to gather information from various sources—a critical skill in clinical care.

Recognizing this challenge, the School of Medicine has formed an AI working group. Their goal is to explore curriculum updates that incorporate AI tools to enhance student learning without making them overly reliant on AI.

Alicia DiGiammarino expressed her concerns, stating, "We don't want doctors who were so reliant on AI at school that they failed to learn how to reason through cases on their own. But I'm more scared of a world where doctors aren't trained to effectively use AI and find it prevalent in modern practice."

Eric Strong added, "We may be decades away from anything like the wholesale replacing of doctors, but we're only a few years away from having to incorporate AI into everyday medicine."

Embracing AI with HYPS Tutors for Ethical Learning

As AI reshapes the landscape of education and medical practice, ethical learning becomes paramount. With HYPS tutors, students have access to valuable insights and guidance to harness state-of-the-art technologies like AI effectively. HYPS is here to facilitate ethical and responsible AI utilization, ensuring that students maximize their potential while upholding the highest standards of integrity and patient care.

With HYPS, you can navigate the evolving world of AI in education and embark on a journey to achieve your educational and ethical goals.