# Project #project/AIbasedlearning #note/sourcereview/article ## Topics: #on/generativeai #on/standardizedpatients #on/virtual ### Source: Maicher, K. R., Stiff, A., Scholl, M., White, M., Fosler-Lussier, E., Schuler, W., Serai, P., Sunder, V., Forrestal, H., Mendella, L., Adib, M., Bratton, C., Lee, K., & Danforth, D. R. (2023). Artificial intelligence in virtual standardized patients: Combining natural language understanding and rule based dialogue management to improve conversational fidelity. _Medical Teacher_, _45_(3), 279–285. [https://doi.org/10.1080/0142159X.2022.2130216](https://doi.org/10.1080/0142159X.2022.2130216) %% ### Hookmark Link: [Maicher et al. - 2023 - Artificial intelligence in virtual standardized pa.pdf](hook://file/RmDBdtRv0?p=c3RvcmFnZS9LSTVYQzdOMg==&n=Maicher%20et%20al%2E%20%2D%202023%20%2D%20Artificial%20intelligence%20in%20virtual%20standardized%20pa%2Epdf) %% ### Notes: “ChatScript is primarily a pattern matching system that is well suited to doctor–patient interactions which tend to be relatively narrow in scope but involve in-depth nuanced conversations. Overall accuracy with this system was 70–85% (Maicher et al. 2017, 2019).” (Maicher et al., 2023, p. 2) “Preliminary retrospective data from our lab indicated that combining ChatScript with a Natural Language Understanding (NLU) system using convolutional neural network (CNN) classifiers might improve conversational accuracy in chatted (typed) interactions (Jaffe et al. 2015;Gokcenetal.2016;Jinetal.2018; Maicher et al. 2019).” (Maicher et al., 2023, p. 2) “In the present study, we examine whether combining ChatScript with a Neural Network system to manage student-VSP dialogue in real time can improve dialogue accuracy in spoken conversations.” (Maicher et al., 2023, p. 2) “We analyzed the data from 620 students who were in Med 1 during 2018, 2019, and 2021. A technical failure of the VP app precluded analyzing the data from the 2020 class. Each class was analyzed separately. For the 2018 and 2019 classes we employed a convolutional neural network to analyze question inputs. For the 2021 class we employed a recurrent neural network classifier. The specifics of each are described below.” (Maicher et al., 2023, p. 2) “The average score for this activity was 3.32 ± 1.16 which was below the average for all learning activities for this cohort of students (4.18 ± 0.53). Review of student comments revealed that some students had challenges getting the app to work on their iPads but the majority felt that the Virtual Patient did a good job of answering their questions and that the exercise was a worthwhile opportunity to practice their history-taking skills.” (Maicher et al., 2023, p. 5) “The accuracy of any system depends in part on the complexity of the case, type of history obtained, the skill of the student, and the nature of the dialogue. In the present study, we had a relatively straightforward case, but the students were early learners who were unskilled at taking a history. As such their questions were not focused and often consisted of non-standard language and included multiple questions in the same utterance. In addition, these were spoken dialogues as opposed to chatted (typed) conversations which result in longer and more complex utterances, and which included numerous instances of irrelevant questions and question fragments. Despite these challenges, the system was able to provide an acceptable response approximately 90% of the time demonstrating the effectiveness of our dialogue approach and the potential of automated, dialogue-driven VSP encounters. Student satisfaction with the system was positive, and most students indicated that it was a useful approach to practicing their initial history taking skills.” (Maicher et al., 2023, p. 6)