The use of large language model-based chatbots in promoting behavior change has been explored by researchers at the University of Illinois Urbana-Champaign, who found that the artificial intelligence tools struggle to recognize certain motivational states of users. Michelle Bak and Jessie Chin from the ACTION Lab conducted a study to assess how well large language models can identify motivational states and provide relevant information to support behavior change. The researchers evaluated various scenarios targeting health needs such as physical activity, diet, mental health challenges, and cancer screening. They found that while the models can support users who have already committed to taking action, they struggle to recognize and provide appropriate information to guide users in the earlier stages of behavior change where hesitancy or ambivalence may exist.
Chin explained that the language models are trained to represent the relevance of a user’s language, but they struggle to understand the difference between a user who is thinking about change but is hesitant and one who is ready to take action. The models also struggle to discern the different motivational states based on how users generate their queries, making it challenging to provide appropriate information in the initial stages of behavior change. When users are resistant to habit change, the large language models fail to provide information that could help them evaluate their problem behavior and its causes, consequences, and environmental influences.
Once a user has made the decision to take action, the large language models are able to provide adequate information to move them toward their goals. However, the models do not effectively provide information on using a reward system to maintain motivation or reducing stimuli in the environment that may lead to a relapse of problem behavior. The researchers found that the chatbots primarily focus on providing external help, such as social support, but lack information on how to control the environment to eliminate stimuli that reinforce problem behavior.
The study concluded that large language model-based chatbots are not yet able to accurately recognize motivational states in natural language conversations but have the potential to support behavior change when users have strong motivations and readiness to take action. Future studies will explore how to improve the models by using linguistic cues, information search patterns, and social determinants of health to better understand users’ motivational states. Additionally, providing the models with more specific knowledge on behavior change strategies may enhance their ability to assist users in making healthy changes to their behavior.