Educational chatbots for project-based learning: investigating learning outcomes for a team-based design course Full Text

Moreover, the complexity of designing and capturing all scenarios of how a user might engage with a chatbot also creates frustrations in interaction as expectations may not always be met for both parties (Brandtzaeg & Følstad, 2018). Hence, while ECs as conversational agents may have been projected to substitute learning platforms in the future (Følstad & Brandtzaeg, 2017), much is still to be explored from stakeholders’ viewpoint in facilitating such intervention. These automated conversational agents (Riel, 2020) have been significantly used to replicate customer service interaction (Holotescu, 2016) in various domains (Khan et al., 2019; Wang et al., 2021) to an extent it has become a common trend (Wang et al., 2021). To this end, initial codes were identified by open coding and iteratively improved through comparison, group discussion among the authors, and subsequent code expansion. Further, codes were supplemented with detailed descriptions until a saturation point was reached, where all included studies could be successfully mapped to codes, suggesting no need for further refinement. As an example, codes for RQ2 (Pedagogical Roles) were adapted and refined in terms of their level of abstraction from an initial set of only two codes, 1) a code for chatbots in the learning role and 2) a code for chatbots in a service-oriented role.

A systematic review follows a rigorous methodology, including predefined search criteria and systematic screening processes, to ensure the inclusion of relevant studies. This comprehensive approach ensures that a wide range of research is considered, minimizing the risk of bias and providing a comprehensive overview of the impact of AI in education. Firstly, we define the research questions and corresponding search strategies and then we filter the search results based on predefined inclusion and exclusion criteria. Secondly, we study selected articles and synthesize results and lastly, we report and discuss the findings.

challenges and risks of chatbots

The chatbots studied in the current literature are traditional, FAQ-type chatbots. Historically, educators viewed their interactions with chatbots negatively, citing that the responses from chatbots were rigid and unoriginal (Kim & Kim, 2022), only capable of answering simple questions (Cunningham-Nelson et al., 2019). This is likely because these traditional chatbots, or frequently asked questions (FAQ)-type chatbots (Cunningham-Nelson et al., 2019; Merelo et al., 2022), do not utilize AI and are trained to respond using predetermined criteria (Smutny & Schreiberova, 2020).

The application of the CHISM model in the evaluation of four AICs has provided valuable insights into the effectiveness of these tools in language learning. The model, which comprises three dimensions (LEX, DEX, UEX), has allowed for a comprehensive assessment of the AICs across multiple facets. This suggests that while these tools have made strides in providing language-related features, there is still room for improvement, particularly in terms of maintaining contextually relevant dialogues and varying sentence complexity based on the learner’s level. This line of research investigates how the interactive nature of some AICs can reduce students’ anxiety and cognitive load (Hsu et al., 2021) and promote an engaging learning environment (Bao, 2019).

Learning performance for the course

The information you collect can help determine whether the customer should purchase through self-service or your sales team, and it can identify which agent should hop into the conversation. A chatbot lets you cut the wait time and interact with your customers immediately. This means you can answer questions or start collecting the information your human agents need to address customer queries faster. These new tools represent Dx 2.0 examples that will transform the experiences of our students and the information and insights to which they have access. To make these transformations happen requires intentional shifts in technology, culture, and workforce. A recent blog published by Cambridge shared research on how English teachers around the world are using generative AI tools.

Moreover, according to Cunningham-Nelson et al. (2019), one of the key benefits of EC is that it can support a large number of users simultaneously, which is undeniably an added advantage as it reduces instructors’ workload. Colace et al. (2018) describe ECs as instrumental when dealing with multiple students, especially testing behavior, keeping track of progress, and assigning tasks. Furthermore, ECs were also found to increase autonomous learning skills and tend to reduce the need for face-to-face interaction between instructors and students (Kumar & Silva, 2020; Yin et al., 2021).

Pedagogical Roles

Participation was voluntary, and students who actively engaged with the chatbots and completed all tasks, including submitting transcripts and multiple-date screenshots, were rewarded with extra credits in their monthly quizzes. This approach ensured higher participation and meaningful interaction with the chatbots, contributing to the study’s insights into the effectiveness of AICs in language education. Some studies have emphasized that interactions with AICs can seem detached and lack the human element (Rapp et al., 2021). Additionally, while AICs can handle a wide range of queries, they may struggle with complex language nuances, which could potentially lead to misunderstandings or incorrect language usage. It has also been observed that some students’ interest dwindled after the initial period of engagement due to repetitive conversation patterns and redundancies, making the interaction less natural compared to student–teacher exchanges (Fryer et al., 2019).

To structure research topics and findings in a comprehensible way, a three-stage clustering process was applied. While the first stage consisted of coding research topics by keywords, the second stage was applied to form overarching research categories (Table 1). In the final stage, the findings within each research category were clustered to identify and structure commonalities within the literature reviews. For instance, some teaching agents recommended tutorials to students based upon learning styles (Redondo-Hernández & Pérez-Marín, 2011), students’ historical learning (Coronado et al., 2018), and pattern matching (Ondáš et al., 2019). In some cases, the teaching agent started the conversation by asking the students to watch educational videos (Qin et al., 2020) followed by a discussion about the videos.

Enhanced student engagement through chatbot interactions

Every chatbot is different, and depends largely on how much content you put in and how robust a conversation you want to design. As a rule of thumb, it takes one person about a month to make a chatbot with 30 different outputs (ie, types of content you want the user to engage with). In the images below you can see two sections of the flowchart of one of my chatbots. In the first one you can see that the chatbot is asking the person how they are feeling, and responding differently according to their answer. By leveraging this valuable feedback, teachers can continuously improve their teaching methods, ensuring that students grasp concepts effectively and ultimately succeed in their academic pursuits. With artificial intelligence, the complete process of enrollment and admissions can be smoother and more streamlined.

In addition to the courses recommended above for educating teachers and students on practical uses of chatbots, hands-on courses should be developed for teachers and students on how to use the technology. Although chatbot technology is novel, PEU may increase over time as the public becomes more accustomed to using the technology. Next, it was interesting to observe the differences and the similarities in both groups for teamwork. In the EC group, there were changes in terms of how students identified learning from other individual team members towards a collective perspective of learning from the team.

Chatbots for Education

However, there have been contradictory findings related to critical thinking, learning engagement, and motivation. Deng and Yu (2023) found that chatbots had a significant and positive influence on numerous learning-related aspects but they do not significantly improve motivation among students. Contrary, Okonkwo and Ade-Ibijola (Okonkwo & Ade-Ibijola, 2021), as well as (Wollny et al., 2021) find that using chatbots increases students’ motivation. Due to AI integration in benefits of chatbots in education the workplace, the World Economic Forum (2023) estimates that by 2027, 25% of companies expect job loss, while 50% expect job growth. In May 2023, Google (2023) and Microsoft (Panay, 2023) announced that their products would integrate AI. As chatbots become more popular and AI becomes increasingly integrated into day-to-day life, it is important to prepare students for the future, as skills using these technologies may be a requirement when entering the future workforce.

Subsequently, a statistical analysis was conducted to evaluate the impact of language nativeness (Spanish and Czech versus non-Spanish and non-Czech speakers), revealing no significant differences in the study’s outcomes. Furthermore, the evaluations of the AICs by both Spanish and Czech cohorts displayed similar results. This analysis led us to conclude that language nativeness and the specific educational settings of the participants were not key factors influencing the results of our study.

Example feedback is such as “I learn to push myself more and commit to the project’s success.” Nevertheless, in both groups, all the trends are almost similar. This question reflects on providing feedback on one aspect they have learned from their team that they probably would not have learned independently. 6, the illustration describes changes in each group (EC and CT) pre and post-intervention. First, teamwork showed an increasing trend for EC, whereas CT showed slight changes pre and post-intervention. This work was supported by the Ministry of Higher Education, Scientific Research and Innovation, the Digital Development Agency (DDA), and the CNRST of Morocco (Al-Khawarizmi program, Project 22). Authors are thankful to all the teaching staff from the Regional Center for Education and Training Professions of Souss Massa (CRMEF-SM) for their help in the evaluation, and all of the participants who took part in this study.

Since the release of ChatGPT, several chatbots and LLMs have been released, such as Bing (Microsoft, 2023), LLaMA (Touvron et al., 2023), Google Bard (Pichai, 2023, February 6), Alpaca (Taori et al., 2023), Vicuna (Chiang et al., 2023) and Guanaco (Dettmers et al., 2023). It is expected that as these models become more widely available for commercial use, research on the benefits of their use will also increase. Because chatbots using LLMs have vastly more capabilities than their traditional counterparts, it is expected that there are additional benefits not currently identified in the literature. Therefore, this section outlines the benefits of traditional chatbot use in education. Given these results, we can summarize four major implementing objectives for chatbots. Of these, Skill Improvement is the most popular objective, constituting around one-third of publications (32%).