Linguistic Indicators of Symptom Severity in Mental Health: From Lexicons to Large Language Models

Language is the medium through which both assessment and therapy are conducted in the care of mental health conditions. The rapid advancement of natural language processing (NLP) technologies presents new avenues for automated appraisal of patient-generated language to support care delivery. This talk will provide an overview of an extended program of research focused on automated appraisal of language across a range of mental health conditions - from assessment of speech in psychosis through detection of suicide risk in internet search logs - with a focus on the relationship between the constraints and capabilities of evolving NLP technologies, and their utility in characterizing the linguistic manifestations of mental health conditions.

Dr. Cohen trained and practiced as a physician in South Africa before earning a PhD in Biomedical Informatics from Columbia University in 2007. His doctoral research focused on improving understanding of psychiatric clinical text using computational models language. He later held faculty positions at Arizona State University and the University of Texas School of Biomedical Informatics, where he worked on representing and applying knowledge extracted from the biomedical literature, amongst other research topics. Since joining the University of Washington in 2018, his work has focused on detecting linguistic indicators of neurocognitive and mental health status, summarizing biomedical literature in plain language, and addressing sources of inaccuracy in clinical language models. Dr. Cohen’s work has been supported by the National Library of Medicine and the National Institute of Mental Health, amongst other sources. He is a Fellow of the American College of Medical Informatics, lead editor of a textbook on AI in medicine, and co-author of a recent book on large language models.

Beyond Helpful: Social Competence and Human Interaction in LLMs

Large language models are increasingly used in settings that look less like abstract benchmark tasks and more like real social interactions. In many of those domains, language technologies can shape trust, self-perception, access, and harm in uneven ways across users. In this talk, I will argue that the success of LLMs does not mean the core problems of NLP have disappeared. Instead, many of them have returned in new forms: as socio-technical challenges around safety, bias, evaluation, and unequal usefulness across people with different backgrounds, needs, and resources. Drawing on recent work, I will discuss what happens when we move from evaluating models on fixed benchmarks to studying how they interact with people in the real world. A central theme will be that better language models are not necessarily better social actors. LLMs can appear helpful while failing some users, sound aligned while flattening disagreement, or perform well on average while being unreliable in the cases that matter most. For high-stakes applications, this means that we need evaluation frameworks that take plurality, context, and vulnerability seriously. I will close by suggesting that the next frontier is not only more capable models, but more socially competent and socially accountable ones: systems that adapt appropriately to different users, are evaluated in terms of their real interactional effects, and are designed with a clearer understanding of the human contexts in which they are deployed.

Dirk Hovy is a professor in the Computing Sciences Department, the scientific director of the Data and Marketing Insights research unit, and the current dean for Digital Transformation and AI of Bocconi University. Previously, he was faculty at the University of Copenhagen, got a PhD from USC’s Information Sciences Institute, and a linguistics master’s from Marburg university in Germany. Dirk is interested in what computers can tell us about language and what language can tell us about society. That involves ethical questions of bias and algorithmic fairness in AI. Dirk has authored over 150 articles on these topics, two textbooks on NLP in Python, and forthcoming title at MIT Press. Dirk has co-founded and organized several workshops (on computational social science, and ethics in NLP), was a local organizer for the EMNLP 2017 conference, and general chair of EMNLP 2025. He was awarded an ERC Starting Grant project 2020 for research on demographic bias in NLP. In his spare time, he enjoys cooking, leather working, and picking up heavy things to put them back down.

MD, MA, FACMI, FAMIA, FIAHSI Associate Professor of Biomedical Informatics, Medicine and Psychiatry Vanderbilt University Medical Center https://www.walshscience.com

Dr. Walsh received a degree in Mechanical Engineering from Princeton University and his medical degree from the University of Chicago. He completed residency and chief residency in internal medicine at Columbia University Medical Center and completed postdoctoral training in biomedical informatics at Columbia University. Dr. Walsh joined the faculty at Vanderbilt University in 2015. Dr. Walsh’s research focuses on machine learning to enable clinical decision support, scalable phenotyping using structured and unstructured clinical data, and public health informatics. His work is nationally recognized for advancing data-driven approaches to injury, overdose, and violence prevention, including the development and real-world implementation of predictive models to prevent suicide and opioid overdose in health systems and population settings. He has led and contributed to multiple federally funded studies and pragmatic clinical trials that translate artificial intelligence into deployable interventions for high-risk populations. Across more than one hundred peer-reviewed publications, Dr. Walsh has helped define the methodological and ethical foundations of applying AI to behavioral health and injury prevention, while working with federal agencies, health systems, and national advisory groups to guide implementation at scale. He is a Fellow of the American College of Medical Informatics, the International Academy of Health Sciences Informatics, and AMIA.

Dr. Justin Tauscher is a Research Assistant Professor at the Behavioral Research in Technology and Engineering (BRiTE) Center in the Department of Psychiatry and Behavioral Sciences at the University of Washington. He received his Ph.D. in Counseling and Counselor Education, M.S. in Biomedical Informatics, and a graduate certificate in Implementation Science from the University of Florida. His research uses natural language processing, multimodal smartphone data, and responsible AI methods to identify clinically meaningful patterns in patient-generated language and behavior among individuals with serious mental illness and co-occurring substance use disorders. Dr. Tauscher's broader work integrates digital mental health, biomedical informatics, and implementation science to translate these methods into scalable interventions for real-world behavioral health care. He holds leadership roles on federally funded studies focused on digital phenotyping of hallucination experiences, mobile health implementation, and technology-supported clinician training in community mental health settings. He also leads translational projects developing AI-enabled clinical support tools for telehealth, mobile intervention delivery, and behavioral health workflows. Prior to academia, Dr. Tauscher spent more than a decade in community behavioral health as a dually licensed mental health and addiction counselor, clinical supervisor, and program director. This clinical experience continues to shape his research agenda, which emphasizes practical, ethically grounded technologies that can expand access to care, support clinicians, and improve outcomes for people with complex behavioral health needs.

Philip Resnik is a Professor at the University of Maryland with joint appointments in the Department of Linguistics and the Institute for Advanced Computer Studies. In 2020 he was named an ACL Fellow for significant contributions to symbolic-statistical methods for natural language processing, multilinguality, and the interdisciplinary study of language. Philip’s most recent research has focused in three main areas. The first is computational social science, with an emphasis on qualitative analysis and connecting the signal available in people’s language use with underlying mental state. The second is the computational cognitive neuroscience of language, using computational modeling in connection with brain imaging to look at the role of context and predictive processing during online language comprehension. The third involves fundamental questions about how current AI models relate to human cognition and to human society. Outside academia, Philip’s industry experience includes research at Bolt Beranek and Newman and Sun Microsystems Laboratories, as well as an internship at IBM T.J. Watson Research Center, and in entrepreneurial life he has been a technical co-founder of CodeRyte (clinical NLP, acquired by 3M in 2012), an advisor to FiscalNote (machine learning and analytics for government relations, went public in 2022), and he currently serves as an advisor to Trustible (technology provider for responsible AI governance). Philip was an undergrad in CS at Harvard and earned his PhD in Computer and Information Science at the University of Pennsylvania.

Anat Brunstein Klomek is a clinical psychologist and Dean of the Baruch Ivcher School of Psychology at Reichman University. Her research focuses on resilience and suicide prevention, with extensive work in the fields of trauma and technology in mental health. She developed a transdiagnostic Interpersonal Psychotherapy (IPT) approach for the treatment of trauma, depression, and anxiety. Her work integrates research, clinical practice, and innovation to advance accessible, evidence-based mental health care.