From Reddit to ChatGPT: A Shift in How People Seek and Offer Advice Online

Written By:

Professor Hahn Jungpil
Vice Dean, Communications
Director, NUS Fintech Lab
Deputy Director (AI Governance), AI Singapore
Deputy Director, Centre for Technology, Robotics, Artificial Intelligence & the Law

Dr Gao Yuting
Research Fellow, AI Singapore

One of the most important technological advancements in recent years is the development of natural language processing (NLP) and the Large Language Models (LLMs). LLMs are trained on large amounts of text data and are able to generate human-like text, answer questions, and complete other language-related tasks with high accuracy. The introduction of Open AI’s chatbot, ChatGPT, on November 30, 2022, has unprecedently increased people’s awareness and exposure to Artificial Intelligence (AI). According to Yahoo Finance, ChatGPT gained 1 million users within one week and reached 100 million monthly active users just two months after launch.

Although LLMs such as ChatGPT are relatively new technologies, and many people including AI experts are still skeptical about their capabilities and potential risks, ChatGPT is becoming more and more popular among the public as it can respond to a wide variety of prompts in various different contexts in a sophisticated and nuanced way. For instance, students have used ChatGPT to assist with their studies, assignments, and exams, for answering questions and essay writing; companies and businesses can use ChatGPT to generate customized promotion emails and provide prompt replies to customers’ inquiries and requests. ChatGPT can also be used to provide emotional support and mental health services to individuals in need. For example, many people voluntarily use ChatGPT to receive general life advice and even for mental therapy instead of seeing therapists. On the other hand, mental health service providers have adopted GPT-based AI to generate responses to questions or problems posted by users. Koko, a non-profit online peer support platform, found that compared to responses generated by other human users, the responses generated by GPT-based AI models were rated better in quality by the users who asked for advice or help.

Previous AI-based chatbots are usually designed and trained for a specific task, such as product recommendation, education, and customer service. Therefore, researchers have typically focused on the chatbot itself and explored topics such as chatbot design and development, adoption and application, and user-chatbot interaction. Little research has explored the externality of chatbots, for example, how chatbots’ adoption and usage may influence users’ engagement with other services. This study aims to fill the research gap and answer the research question: if people can use ChatGPT to get advice and support, how will people’s engagement with other online advice communities be influenced?

On the one hand, ChatGPT can respond to users’ queries and provide advice and support on a wide range of topics for free 24/7. Under such circumstances, ChatGPT may offer advice more promptly than online advice communities, where it takes time for users to respond to a request. On the other hand, ChatGPT is a general-purpose LLM and is not specially trained for offering advice, so it may make incorrect or potentially harmful statements. For example, a Belgian man committed suicide after a chatbot “encouraged” him to sacrifice his life to stop climate change. Therefore, it may be risky for people to discuss personal issues with ChatGPT.

To explore the impact of ChatGPT on people’s engagement with online advice communities, we consulted Reddit, one of the largest online communities, and analyzed data of 20 subreddits intended for users to seek advice from and offer advice to others. We found that after the release of ChatGPT, users’ posting and commenting behaviors on these subreddit communities significantly dropped. In addition, the drop of users’ posting behavior is weaker on subreddits related to health and wellness, while the drop of users’ commenting behavior is stronger on subreddits related to health and wellness. Being one of the first studies exploring the effect of ChatGPT on people’s online advice-seeking and offering behaviors, this work contributes to the literature on the broader societal impacts of AI and technology on social interactions. Practically, considering the increasing adoption of AI in many areas of society, our finding suggests that AI can lead to unintended negative consequences on people’s social behaviors and interpersonal interactions. More studies should be done to explore the psychological and social impact of ChatGPT and AI before they are widely implemented and adopted.

To bring our work to full fruition, first, we plan to conduct a survey to know whether individuals have used or intend to use ChatGPT when they need advice on their personal issues, such as career development, relationship and dating, and health and wellness. This can corroborate our empirical findings on Reddit. Next, we plan to conduct a laboratory experiment, to study users’ perceived ease of use and perceived usefulness of ChatGPT (e.g., perceived response quality of ChatGPT), and attitudes towards ChatGPT, as an advice seeker and as an advice giver, on various topics, especially topics related to health and wellness. This will enhance our understanding of the reason why users may switch from online communities to ChatGPT. Theoretically, users are likely to perceive a higher level of perceived ease of use of ChatGPT because it is accessible 24/7 and provides quick responses. However, it is uncertain if the perceived usefulness of ChatGPT is indeed higher than Reddit or not. The perceived usefulness of advice-seekers and advice-givers may help explain why there are differences in the moderation effect of the type of subreddit community. In addition, we plan to collect the original posts and comments from these subreddits, to qualitatively analyze the texts. It is possible that the release of ChatGPT may also affect the wording of posts and comments in addition to the number of posts and comments. For example, before posting a question on Reddit, a user may use ChatGPT to help her frame the question. Or, before commenting on a post on Reddit, a user may use ChatGPT to help her improve her response. A qualitative analysis of the posts and comments in subreddit communities can enhance our understanding of the effect of ChatGPT on online advice posts, for instance, in terms of vocabulary, style of writing, and phrasing and wording.

Please refer to the article.