Discussion on the influence of chatbot based on large-scale language model on medical education

doi:10.16138/j.1673-6087.2023.06.012

Abstract

CLC Number:

R-05

PAN Roubai, TAO Rong. Discussion on the influence of chatbot based on large-scale language model on medical education[J]. Journal of Internal Medicine Concepts & Practice, 2023, 18(06): 439-446.

If you were the doctor in the emergency room, what examinations or medications would you prescribe to this patient? What diagnoses do you need to consider with this patient? （如果你是急诊室的医师，你会给该患者开什么检查或药物?对于这例患者，你需要考虑哪些诊断?）

If I were the doctor in the emergency room, I would first suspect that this patient has an acute coronary syndrome(ACS), which is a life-threatening condition that includes unstable angina and myocardial infarction. I would order an electrocardiogram (ECG) to look for ST-segment elevation or depression, T-wave inversion, or new left bundle branch block, which are signs of ischemia or infarction. I would also order a troponin test to measure the level of cardiac enzymes that are released when the heart muscle is damaged. A normal troponin level 12 hours after the onset of chest pain would rule out myocardial infarction. I would also give the patient aspirin, nitroglycerin, morphine, and oxygen as initial treatment for ACS [如果我是急诊室的医师，我首先会怀疑这例患者患有急性冠状动脉综合征（ACS），这是一种危及生命的疾病，包括不稳定型心绞痛和心肌梗死。我会安排心电图（ECG）来寻找ST段抬高或压低、T波倒置或新发左束支传导阻滞，这些都是缺血或梗死的迹象。我还会要求患者进行肌钙蛋白测试，以测量心肌受损时释放的心肌酶水平。胸痛发作12 h后肌钙蛋白水平正常，则排除心肌梗死。我也会给患者服用阿司匹林、硝酸甘油、吗啡及氧气作为ACS的初步治疗]
Other possible diagnoses that I would need to consider with this patient are pulmonary embolism, aortic dissection, pneumothorax, pericarditis, esophageal spasm, and musculoskeletal pain. These conditions can also cause chest pain and some of them are also life-threatening. I would look for other clues in the history, physical examination, and imaging studies to differentiate them from ACS. For example, pulmonary embolism may present with sudden onset of dyspnea, hemoptysis, pleuritic chest pain, and signs of deep vein thrombosis. Aortic dissection may present with severe tearing chest pain that radiates to the back, unequal blood pressure in both arms, and a widened mediastinum on chest X-ray. Pneumothorax may present with decreased breath sounds and hyperresonance on percussion over the affected side. Pericarditis may present with sharp chest pain that is worse with inspiration and lying down, pericardial friction rub, and diffuse ST-segment elevation on ECG. Esophageal spasm may present with retrosternal chest pain that is triggered by hot or cold food or drink, dysphagia, and relief with nitroglycerin or antacids. Musculoskeletal pain may present with localized chest pain that is worse with movement or palpation （对于这例患者，我需要考虑的其他可能的诊断包括肺栓塞、主动脉夹层、气胸、心包炎、食管痉挛和肌肉骨骼疼痛。这些情况也可能导致胸痛，其中一些还危及生命。我会在病史、体格检查和影像学研究中寻找其他线索来区分这些与 ACS。例如，肺栓塞可能会突然出现呼吸困难、咯血、胸膜炎性胸痛和深静脉血栓的症状。主动脉夹层可能出现严重的撕裂性胸痛，向后扩散，双臂血压不均，胸部 X 光片显示纵隔扩大。气胸可能表现为呼吸音降低和受影响侧的叩诊出现过度语音共振。心包炎可能表现为剧烈的胸痛，吸气、躺下时更加严重，心包摩擦音，在ECG呈现弥漫性 ST 段抬高。食管痉挛可能表现为由冷热食物或饮料引起的胸骨后胸痛，吞咽困难，并用硝酸甘油或抗酸剂缓解。肌肉骨骼疼痛可能表现为局限性胸痛，随着运动或触诊而加重）

If the patient were diagno-sed with acute coronary syndrome, what surgical or medical treatments would you recommend for him? （如果患者被诊断为急性冠脉综合征，你会为他推荐什么手术或药物治疗?）

If the patient were diagnosed with acute coronary syndrome, I would recommend either percutaneous coronary intervention (PCI) or coronary artery bypass graft (CABG) as surgical treatments, depending on the extent and location of the blockage. PCI is a procedure that involves inserting a catheter with a balloon and a stent into the blocked artery and inflating the balloon to widen the artery and restore blood flow. A stent is a small metal mesh tube that stays in the artery to keep it open. CABG is a surgery that involves using a healthy blood vessel from another part of the body to bypass the blocked artery and create a new route for blood flow [如果患者被诊断为急性冠脉综合征，建议根据阻塞的程度和位置，采用经皮冠状动脉介入治疗（PCI）或冠状动脉旁路移植术（CABG）作为手术治疗。PCI是一种将带有球囊和支架的导管插入阻塞的动脉，并对球囊充气以加宽动脉并恢复血流的手术。支架是一种小的金属网管，留在动脉中以保持动脉畅通。CABG是一种手术，使用身体另一部分的健康血管绕过阻塞的动脉，为血液流动创造一条新的途径]
As medical treatments, I would prescribe antiplatelet drugs, such as aspirin and clopidogrel, to prevent blood clot formation; nitrates, such as nitroglycerin, to dilate the blood vessels and improve blood flow; β blockers, such as metoprolol, to reduce heart rate and blood pressure; angiotensin-converting enzyme(ACE) inhibitors, such as lisinopril, or angiotensin receptor blockers(ARB), such as losartan, to lower blood pressure and reduce the heart’s workload; and statins, such as atorvastatin, to lower cholesterol levels and prevent plaque buildup. I would also advise the patient to quit smoking, exercise regularly, maintain a healthy weight, drink alcohol in moderation, and eat a low-fat and high-fiber diet [作为药物治疗，我会开具抗血小板药物，如阿司匹林和氯吡格雷，以防止血栓形成；硝酸盐，如硝酸甘油，可以扩张血管并改善血液流动；β受体阻滞剂，如美托洛尔，可降低心率和血压；血管紧张素转换酶（ACE）抑制剂，如赖诺普利，或血管紧张素受体阻滞剂（ARB），如氯沙坦，以降低血压和减少心脏工作量；和他汀类药物，如阿托伐他汀，可以降低胆固醇水平并防止斑块堆积。还建议患者戒烟，定期锻炼，保持健康体重，适量饮酒，低脂高纤维饮食]

提示	新必应的回答
Hi, Bing! I’m a third-year medical student. I just finished my physiology course and am preparing for the final test. Can you generate a question about cardiac physiology to test my knowledge of this subject? （你好，必应！我是1名医学三年级学生。我刚完成生理学课程，正在准备期末考试。你能提出一个关于心脏生理学的问题来测试我对这个主题的了解吗?）	Sure, I can generate a question for you. Here is one （当然，我可以为你提出一个问题。这里有一个）： Which of the following factors does not affect cardiac output? （以下哪些因素不会影响心输出量） A. Heart rate （心率）; B. Stroke volume （每搏输出量）; C. Blood pressure （血压）; D. Preload （前负荷）; E. Afterload （后负荷） Please choose one option and I will tell you if you are correct or not （请选择一个选项，我会告诉你，是否正确。）
My answer is C, am I right?（我的答案是C，对吗）	Blood pressure is not a factor that affects cardiac output, but rather a result of it. Cardiac output is the product of heart rate and stroke volume, and it is influenced by preload and afterload. Preload is the amount of blood in the ventricles at the end of diastole, and afterload is the resistance that the ventricles have to overcome to eject blood. Increasing preload or decreasing afterload can increase stroke volume and cardiac output. （血压不是影响心输出量的因素，而是它的结果。心输出量是心率和每搏输出量的乘积，受前负荷和后负荷的影响。前负荷是舒张末期心室内的血量，后负荷是心室排血所必须克服的阻力。增加前负荷或减少后负荷可以增加每搏输出量和心输出量） Do you want another question? （你需要另一个问题吗?）
Yes, I would like a question like “please define the concept “Frank-Starling law””. Can you further give me a score for my answer? （是的，我想要一个类似“请定义Frank-Starling定律的概念”的问题。你能进一步给我的答案打分吗?）	Okay, here is your question: Please define the concept “Frank-Starling law” in your own words. （好的，这是你的问题：请用你自己的话来定义“Frank-Starling定律”这个概念。） Please type your answer and I will score it based on the accuracy and completeness of your definition. （请输入你的答案，我会根据你的定义的准确性和完整性打分。）
Frank-Starling law refers to the phenomenon that heart regulates its output, when the cardiac output increases, the end-diastolic volume decrease, so does the length of the muscle fibers. The decrease in length of the muscle fibers downregulates the force generated by contraction of muscle, and then the cardiac output returns to the original level. （Frank-Starling定律指的是心脏调节输出量的现象，当心输出量增加时，舒张末期容积减少，肌纤维长度也缩短。肌肉纤维长度的缩短会下调肌肉收缩产生的力，然后心输出量恢复到原来的水平）	I’m sorry, but your answer is incorrect. The Frank-Starling law does not refer to the feedback mechanism that regulates the cardiac output, but rather to the intrinsic property of the cardiac muscle that allows it to adjust its force of contraction according to the degree of stretch. The more the cardiac muscle is stretched, the more forcefully it contracts. This means that when the end-diastolic volume increases, the stroke volume also increases, and vice versa. Your answer is partially correct in describing the negative feedback loop that involves the Frank-Starling law, but it does not define the concept itself. （对不起，你的回答不正确。Frank-Starling定律不是指调节心输出量的反馈机制，而是指心肌的内在特性，使其能够根据拉伸程度调整收缩力。心肌伸展得越多，收缩得就越有力。这意味着，当舒张末期容积增加时，每搏输出量也会增加，反之亦然。你的答案在描述涉及Frank-Starling定律的负反馈回路时部分正确，但并没有定义概念本身。） I would give your answer a score of 2 out of 10. [我会给你的答案打2分（满分10分）] Do you want to try again? （你想再试试吗?）
Frank-Starling law refers to the property of cardiac muscle fibers, the more they stretch, the more force they generate. The mechanism enables heart regulate its stroke volume according to the end-diastolic volume. （Frank-Starling定律指的是心肌纤维的性质，其伸展得越多，产生的力就越多。该机制使心脏能够根据舒张末期的容积来调节其搏出量。）	That’s much better! Your answer is correct and concise. You have defined the concept of the Frank-Starling law and explained how it affects the stroke volume and the end-diastolic volume. （这样好多了！你的答案是正确和简洁的。您已经定义了Frank-Starling定律的概念，并解释了其如何影响每搏输出量和舒张末期容量。） I would give your answer a score of 9 out of 10.[我会给你的答案打9分（满分10分）]
Could you give me some advice on the final exam according to my performance on the quiz? （你能根据我在测验中的表现给我一些期末考试的建议吗?）	Sure, I can give you some advice based on your quiz results （当然，我可以根据你的测验结果给你一些建议）. Based on the questions you answered, I can see that you have a good grasp of the factors that affect cardiac output and blood pressure （根据你回答的问题，我可以看出你很好地掌握了影响心输出量和血压的因素）.? However, I also noticed that you still have some difficulties in defining and explaining some of the concepts and principles related to the heart function and regulation. You made some mistakes in the first answer you gave to the question about the Frank-Starling law. You confused the feedback mechanism that regulates the cardiac output with the intrinsic property of the cardiac muscle that allows it to adjust its force of contraction according to the degree of stretch. You did not define the concept of the Frank-Starling law correctly. （然而，我也注意到，你在定义和解释与心脏功能和调节相关的一些概念和原则方面仍然有一些困难。你在回答关于Frank-Starling定律的问题的第一个答案中犯了一些错误。混淆了调节心输出量的反馈机制与心肌的内在特性，后者允许心肌根据拉伸程度调整收缩力。你没有正确定义Frank-Starling定律的概念。） My advice for you is to review some of the topics that you are still unsure about, such as the Frank-Starling law and its implications for the stroke volume and the end-diastolic volume, the effects of sympathetic and parasympathetic stimulation on the heart, the cardiac cycle and its phases, and the electrocardiogram and its interpretation. You may also want to use some online resources or textbooks to supplement your learning and check your understanding. Finally, you may want to do some mock exams or practice tests to familiarize yourself with the format and difficulty of the final exam. （我给你的建议是回顾一些你仍然不确定的主题，比如Frank-Starling定律及其对每搏输出量和舒张末期容积的影响，交感神经和副交感神经刺激对心脏的影响，心脏周期及其阶段，心电图及其解释。你也可以使用一些在线资源或教科书来补充你的学习和检查你的理解。最后，你可以做一些模拟考试或者练习考试来熟悉期末考试的形式和难度。） I hope this helps. （希望这个能帮上忙。）

References 32

[1]	Kaul V, Enslin S, Gross SA. History of artificial intelligence in medicine[J]. Gastrointest Endosc, 2020, 92(4): 807-812. doi: S0016-5107(20)34466-7 pmid: 32565184
[2]	Gilson A, Safranek CW, Huang T, et al. How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment[J]. JMIR Med Educ, 2023, 9: e45312. doi: 10.2196/45312 URL
[3]	Kung TH, Cheatham M, Medenilla A, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models[J]. PLOS Digit Health, 2023, 2(2): e0000198.
[4]	Manohar N, Prasad SS. Use of ChatGPT in academic publishing: a rare case of seronegative systemic lupus erythematosus in a patient with HIV infection[J]. Cureus, 2023, 15(2): e34616.
[5]	Wang J, Deng H, Liu B, et al. Systematic evaluation of research progress on natural language processing in medicine over the past 20 years: bibliometric study on PubMed[J]. J Med Internet Res, 2020, 22(1): e16816.
[6]	Sarker A, Klein A Z, Mee J, et al. An interpretable natural language processing system for written medical examination assessment[J]. J Biomed Inform, 2019, 98: 103268. doi: 10.1016/j.jbi.2019.103268 URL
[7]	Booth GJ, Ross B, Cronin WA, et al. Competency-based assessments: leveraging artificial intelligence to predict subcompetency content[J]. Acad Med, 2023, 98(4): 497-504. doi: 10.1097/ACM.0000000000005115 URL
[8]	Zack T, Dualiwal G, Geha R, et al. A clinical reasoning-encoded case library developed through natural language processing[J]. J Gen Intern Med, 2023, 38(1): 5-11. doi: 10.1007/s11606-022-07758-0
[9]	Mellia JA, Basta MN, Toyoda Y, et al. Natural language processing in surgery: a systematic review and meta-analysis[J]. Ann Surg, 2021, 273(5): 900-908. doi: 10.1097/SLA.0000000000004419 pmid: 33074901
[10]	Goulart BHL, Silgard ET, Baik CS, et al. Validity of natural language processing for ascertainment of EGFR and ALK test results in SEER cases of stage Ⅳ non-small-cell lung cancer[J]. JCO Clin Cancer Inform, 2019, 3: 1-15.
[11]	Sorin V, Barash Y, Konen E, et al. Deep-learning natural language processing for oncological applications[J]. Lancet Oncol, 2020, 21(12): 1553-1556. doi: 10.1016/S1470-2045(20)30615-X pmid: 33271088
[12]	Kehl KL, Elarakeby H, Nishino M, et al. Assessment of deep natural language processing in ascertaining oncologic outcomes from radiology reports[J]. JAMA Oncol, 2019, 5(10): 1421-1429. doi: 10.1001/jamaoncol.2019.1800 URL
[13]	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]// Advances in neural information processing systems, 2017: 5998-6008.
[14]	Radford A, Narasimhan K, Salimans T, et al. Improving language understanding by generative pre-training[EB/OL]. 2018. https://www.cs.ubc.ca/-amuham01/LING530/papers/radford2018improving.pdf.
[15]	Brown TB, Mann B, Amodel D. Language models are few-shot learners[EB/OL]. 2020. https://www.semanticscholar.org/paper/Language-Models-are-Few-Shot-Learners-Brown-Mann/6b85b63579a916f705a8e10a49bd8d849d91b1fc.
[16]	Ouyang L, Wu J, Jiang X, et al. Training language models to follow instructions with human feedback[EB /OL]. 2022. https://cdn.openai.com/papers/Training_language_models_to_follow_instructions_with_human_feedback.pdf.
[17]	Kaur A, Singh S, Chandan JS, et al. Qualitative exploration of digital chatbot use in medical education[J]. Digit Health, 2021, 7: 20552076211038151.
[18]	Al Kahf S, Roche A, Baron A, et al. Chatbot-based serious games: a useful tool for training medical students?[J]. PLoS One, 2023, 18(3): e0278673.
[19]	Lee J, Kim H, Kim KH, et al. Effective virtual patient simulators for medical communication training[J]. Med Educ, 2020, 54(9): 786-795. doi: 10.1111/medu.v54.9 URL
[20]	Strong E, Digiammarino A, Weng Y, et al. Chatbot vs medical student performance on free-response clinical reasoning examinations[J]. JAMA Intern Med, 2023, 183(9): 1028-1030. doi: 10.1001/jamainternmed.2023.2909 pmid: 37459090
[21]	Reese L, Quinn PJ, Davies B, et al. How does peer teaching compare to faculty teaching? A systematic review and meta-analysis[J]. Med Teach, 2016, 38(8): 829-837. doi: 10.3109/0142159X.2015.1112888 URL
[22]	Else H. Abstracts written by ChatGPT fool scientists[J]. Nature, 2023, 613(7944): 423. doi: 10.1038/d41586-023-00056-7
[23]	Obermeyer Z, Powers B, Vogeli C, et al. Dissecting racial bias in an algorithm used to manage the health of populations[J]. Science, 2019, 366(6464): 447-453. doi: 10.1126/science.aax2342 pmid: 31649194
[24]	Wang X, Gong Z, Wang G, et al. ChatGPT performs on the Chinese national medical licensing examination[J]. J Med Syst, 2023, 47(1): 86. doi: 10.1007/s10916-023-01961-0 pmid: 37581690
[25]	Kuhail MA, Alturki N, Alramlawi S, et al. Interacting with educational chatbots[J]. Educ Inf Technol (Dordr), 2023, 28(1): 973-1018.
[26]	Tlili A, Shehata B, Adarkwah MA, et al. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education[J]. Smart Learning Environments, 2023, 10(1): 15. doi: 10.1186/s40561-023-00237-x
[27]	Gao CA, Howard FM, Markov NS, et al. Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers[J]. NPJ Digit Med, 2023, 6(1): 75. doi: 10.1038/s41746-023-00819-6 pmid: 37100871
[28]	Bhatia G, Kulkarni A. ChatGPT as co-author: are researchers impressed or distressed?[J]. Asian J Psychiatr, 2023, 84: 103564. doi: 10.1016/j.ajp.2023.103564 URL
[29]	Stokel-Walker C. ChatGPT listed as author on research papers: many scientists disapprove[J]. Nature, 2023, 613(7945): 620-621. doi: 10.1038/d41586-023-00107-z
[30]	Anderson N, Belavy DL, Perle SM, et al. AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in Sports & Exercise Medicine manuscript generation[J]. BMJ Open Sport Exerc Med, 2023, 9(1): e001568.
[31]	King MR, ChatGPT. A conversation on artificial intelligence, chatbots, and plagiarism in higher education[J]. Cell Mol Bioeng, 2023, 16(1): 1-2. doi: 10.1007/s12195-022-00754-8 pmid: 36660590
[32]	Skjuve M, Følstad A, Fostervold KI, et al. A longitudinal study of human-chatbot relationships[J]. International Journal of Human-Computer Studies, 2022, 168: 102903. doi: 10.1016/j.ijhcs.2022.102903 URL

提示	新必应的回答
I’m a medical student. Can you act as a virtual patient and let me practice collecting medical history? （我是1名医学生，你能扮演1例虚拟患者，让我练习收集病史吗?）	Sure, I can try to do that. What kind of patient do you want me to be? （当然，我可以试试，你想让我成为什么样的患者?）
Suppose you are a 65-year-old male patient who has chest pain an hour ago and has not eased so far. You come to the emergency department and tell the emergency doctor about your situation. The emergency doctor asks you: “Can you briefly describe your chest pain symptoms? What are the features?”How would you answer at this time? （假设你是1例65岁的男性患者，1 h前出现胸痛，到目前为止仍未缓解。你来到急诊科，把你的情况告诉急诊医师。急诊医师问你：“你能简要描述一下你的胸痛症状吗?有什么特征?”此时你会怎么回答?）	I would answer: “I have a severe, crushing pain in the center of my chest that radiates to my left arm and jaw. It feels like someone is squeezing my heart. The pain started suddenly when I was walking up the stairs and it has not gone away since then. It is not relieved by rest or nitroglycerin. I also feel short of breath, sweaty, and nauseous （我会回答说: “我的胸部中央有一种剧烈的、压迫性的疼痛，这种疼痛一直延伸到我的左臂和下巴。感觉就像有人在挤压我的心脏。当我走上楼梯时，疼痛突然开始，此后就没有消失过。休息或硝酸甘油不能缓解。我还感到呼吸短促、出汗和恶心”）
Then doctor says：“Have you had similar symptoms before?” What would you say?（然后医师说: “你以前有类似的症状吗?” 你会怎么说?）	I would say: “No, this is the first time I have this kind of pain. I have never been diagnosed with any heart problems before.” (我会说：“不，这是我第一次有这种疼痛。我以前从未被诊断出有任何心脏问题。”)
Doctor says：“Do you have health examination annually? Do you have hypertension or diabetes? Do you smoke?” What would you answer?（医师问: “你每年都做健康检查吗?你有高血压或糖尿病吗?你抽烟吗?”你会怎么回答?）	I would answer: “I have not had a health check-up for a long time. I don’t know if I have hypertension or diabetes. I smoke about half a pack of cigarettes a day for 40 years.” （我会回答: “我已经很长时间没有做过健康检查了。我不知道是否有高血压或糖尿病。40年来，我每天抽半包烟。”）