Do Better Next Time: AI Use In Online Learning

I design and deliver an online asynchronous program, Professional English Language Skills (PELS) at University of Toronto Mississauga (UTM). This is a non-credit course that supports academic English language learning and study skills development. Since PELS is supplementary instruction, students can complete low-risk tasks in 8 online modules consisting of videos, written resources, and short assessments. Each module is designed to take an hour, but students may complete each module over seven days.

Since the launch of ChatGPT-OpenAI, students have submitted Artificial Intelligence (AI) generated text in PELS assessments. This trend afforded me the opportunity to teach students about the drawbacks of using AI. Furthermore, I have been able to counsel students that using AI generated text without permission and/or critical thinking will disadvantage them in writing assessment situations.

My poster illustrates the Module 2 and Module 4 paraphrase assessment descriptions. Submissions for Module 2 questions have been almost identical in the Fall 2023 and Winter 2024 offerings of PELS. I posted the quantitative score results for Module 2 with my analysis for students to consider. After I tell students that I have quantitative evidence that strongly suggests most students used AI chatbots to answer the Module 2 questions, I explain why the AI generated text is incorrect and nonsensical, and I caution students about using AI generated text in the Module 4 assessment re: academic integrity violations.

I contend that designing a question that is conducive to AI generated responses in Module 2 presents students with the choice to submit work that is directly copied from an AI chatbot. In doing so, I can reflect students' choices back to them through analysis of quantitative data - e.g. in Winter 2024, 97% of students (n=688) submitted answers that are nearly identical in terms of syntax and lexical elements. The Module 2 responses present a teaching opportunity, and sharing my analysis of students' submissions influences their choices in the Module 4 assessment, a paraphrase task that students must annotate and revise using a different pattern of organization. These tasks help students understand the constraints of uncritical AI use, and it requires that students carefully read the source texts that they are asked to paraphrase. Both paraphrase tasks address linguistic diversity and equity issues to align with the emphasis on critical language awareness and intercultural communicative competence in PELS. In sum, I create an opportunity to cheat - i.e. using AI generated text to answer PELS assessments - and this affords important learning for students. Moreover, it influences students’ choices once I show widespread unauthorized use of AI chatbots in my quantitative analysis of student assessment responses. Framing AI submissions as consistent, incorrect, nonsensical AND an academic integrity violation in Module 2 has been effective for shifting students' choices in Module 4 of PELS.

Bio: Sheila Batacharya https://www.utm.utoronto.ca/isup/people/sheila-batacharya