Article body

The availability and use of artificial intelligence (AI) tools within higher education, including nursing programs, is accelerating significantly (Foronda & Porter, 2024; Sallam, 2023). One such AI tool is ChatGPT version 3.5, which, when prompted, describes itself as

a language model developed by OpenAI called Generative Pre-trained Transformer-3.5. It is the third iteration of the GPT series, known for its advanced natural language processing capabilities. GPT-3.5 is a state-of-the-art language model that has been pre-trained on a diverse range of internet text and can generate coherent and contextually relevant responses to user inputs.

OpenAI, 2024

This study incorporated GPT-3.5 as it was the most recent free version at the time and was available to students at no cost. As ChatGPT proliferated rapidly, with newer versions emerging continuously, many post-secondary institutions initially responded with bans and framed conversations mainly around academic integrity. Despite these institutional responses, adoption of these new technologies persists.

Background

As generative AI technologies are adopted, educators are grappling with both how post-secondary students use these tools and how to address significant uptake in the context of supporting critical thinking and professional growth. Such considerations are especially important within the profession of nursing as these evolving technologies are currently transforming health care systems, with prospective use cases in areas as diverse as forecasting of public health insights for health promotion opportunities, support for clinical decision-making using electronic health record data, and supply chain management (American Nurses Association, 2022). Thus, nurse educators are tasked with not only critically examining the impact of these tools but also incorporating them into nursing curricula. While there is much speculation about the benefits and concerns of ChatGPT in nursing education, a timely scoping review highlights the mostly editorial nature of this literature (Sallam, 2023) and the need for continuous research.

As outlined in the nursing education literature, ChatGPT challenges include negative effects on critical thinking development, especially evaluating research evidence (e.g., Choi et al., 2023); undue trust in notoriously incomplete and/or inaccurate AI tools (e.g., Flanagin et al., 2023; Shen et al., 2023); the introduction of biased and misleading information to student learning and health care (e.g., O’Connor, 2022); and the exacerbation of inequities for those who do not have access (Cotton et al., 2023). By far, the most severe challenge discussed in nursing education is the threat to academic integrity (Choi et al., 2023; Cotton et al., 2023). Academic integrity can be defined as “compliance with ethical and professional principles, standards, practices and consistent system of values, that serves as guidance for making decisions and taking actions in education, research and scholarship” (Tauginienė et al., 2018 pp. 8–9).

A plethora of opportunities exists alongside these challenges. With thoughtful use, AI tools such as ChatGPT may provide opportunities for tailored, learner-centred content and experiences to focus on complex practices (e.g., Irwin et al., 2023; Lim et al., 2022; Neumann et al., 2023). Such use may also be considered to be “future proofing” nursing education, in that AI tools can support educators to provide a learning environment that prepares students for contemporary practice settings (Irwin et al., 2023, p. 2). This preparation is particularly important as AI tools are currently being integrated into practice decision-making tools and electronic health records with the expectation that nurses will use them appropriately and have the skills to discern when the AI-driven tool is not serving the patient in front of them (Mello & Guha, 2024; Swiecki et al., 2022). In addition, the use of AI tools in nursing education potentially improves students’ digital literacy and allows future nurses to meaningfully contribute to the competent use and critical evaluation of digital technologies integrated into health care (Castonguay et al., 2023; Sun & Hoelscher, 2023). Overall, many scholars concur that AI tools hold the potential to assist students to appreciate the principles of academic integrity while promoting their critical thinking (Choi et al., 2023). Further, nurse educators must also understand the benefits and consequences of AI tools to support future nurses with professional writing and the requisite problem-solving skills (Le Lagadec et al., 2024).

Strategies that support nurse educators to capitalize on AI tools are also outlined in the literature. Underscoring these strategies is the recommendation to approach AI tools with caution while also engaging in professional reflection and ongoing evaluation of current pedagogical practices (Castonguay et al., 2023). The main recommendations for faculty include ensuring guiding principles are integrated into course syllabi (Foronda & Porter, 2024); preparing for inevitable discussions regarding ethics and practice challenges; improving technology proficiency; understanding the boundaries of academic integrity; and integrating multimodal learning opportunities using AI tools (e.g., simulation, reflection, collaboration) to demonstrate how new knowledge is applied (Sun & Hoelscher, 2023).

Numerous authors encourage further empirical research to better understand the impacts of ChatGPT on nursing education (Choi et al., 2023; Sallam, 2023). Some scholars have notably studied the use of ChatGPT to generate PICOT (population, intervention, comparison, outcome, time) questions (Branum & Schiavenato, 2023); insights into ChatGPT-assisted critical reflections after a simulated learning activity (Chan et al., 2023); the incorporation of chatbots in simulated emergency scenarios (Rodriguez-Arrastia et al., 2022); the evaluation of ChatGPT-generated care plans (Dağci et al., 2024); and the use of ChatGPT-generated materials for electronic fetal monitoring (Han et al., 2022). One study examining student experiences of ChatGPT case studies, which included a few nursing students, demonstrated the promise of AI-assisted pedagogical tools; however, these authors also recommend caution and further study of educational interventions involving ChatGPT (Tlili et al., 2023). Two other non-nursing studies considered student perceptions. Engineering students enjoyed using ChatGPT to assist with computer coding (Shoufan, 2023). Medical students reported that AI tools will be a career benefit and an important aspect of health care and emphasized that AI tools can never replace physicians (Buabbas et al., 2023). While the perceptions of medical, engineering, and other students provided insights, nursing students’ perceptions regarding the use of AI tools are missing. To address this dearth of information, we conducted an exploration of student perceptions to examine how they both understand and use AI tools for their academic studies.

Shortly before this study, our institution released preliminary guidelines to assist with navigating generative AI tools. Information at that time was vague and focused on discussing academic integrity (particularly plagiarism) and assignment expectations with students. Despite a lack of clarity, instructors were encouraged to consider how to mitigate classroom effects by providing alternative ways for students to demonstrate critical thinking. Since that time, our institution has developed a robust suite of policy tools, including a position statement (University of Victoria, 2024b), a primer for students (University of Victoria, 2024a), and ongoing opportunities for consultation and coaching to promote the ethical and appropriate use of this shifting and unpredictable technology. Our aim is that the findings presented here will support nurse educators in their creative use of AI tools in contemporary academic environments.

The Pedagogical Intervention

Drawing from the findings of empirical studies discussed earlier, we designed an assignment in which students were required to incorporate GPT-3.5 into the creation of their final product as well as critically appraise the GPT-3.5 responses to their prompts. Overall, the mandatory class assignment was the creation of a briefing note regarding a political issue that was meaningful to the student. Detailed instructions for both GPT-3.5 and the assignment expectations were included in the course syllabus. For the assignment, the students were instructed to use their thesis statement as a GPT-3.5 prompt to generate a briefing note. Then, the students used this generated text as an outline to consider and build upon for the final product. The students then responded to three reflective questions: 1) Does the information in the ChatGPT-generated text make sense? Is it credible? Is anything missing? Comment on the quality of the references provided. 2) Did the ChatGPT briefing note frame the policy issue the same way you used in your final version of your assignment? Did ChatGPT make any recommendations that differed from yours? Did this blending improve your learning or expand your understanding of how to frame a policy issue? 3) What were the challenges and/or benefits of using ChatGPT for this assignment? Lastly, the ChatGPT prompt, the generated text, and the student’s responses to the reflective questions were required as appendices to their assignment.

The purpose of the reflection was to encourage students to think critically about both the use and veracity of AI tools in their writing. The assignment was reviewed in detail during the first class, and time was set aside in subsequent classes to answer student questions. Despite explicit and sanctioned use of AI tools for this assignment, students still expressed significant concern about inadvertent plagiarism. To address this concern, extra sessions with the educators were offered to answer student questions and review any written material they were concerned about.

Aim and Research Questions

The aim of the research was to examine nursing students’ perceptions of using ChatGPT version 3.5 within a course assignment. Considering the limited empirical guidance and the absence of institutional policies at the time this research was conducted, we were not evaluating the effectiveness of ChatGPT as a potential pedagogical tool in and of itself. Thus, our overarching research question was the following: How do nursing students perceive the use of ChatGPT before and after completing an assignment with a GPT-3.5 component? As well, we aimed to explore nursing students’ perceptions of GPT-3.5 and similar technologies in relation to their future practice.

Additionally, the aim of the primary researcher (an assistant professor at the time of the study) was to include a doctoral student as well as a practising nurse seeking to return to graduate studies in pedagogical inquiry. We were intrigued by the influence of AI and machine learning tools on nursing education and practice and sought to explore how to better support nursing students as the disruptive force of this technology progresses.

Methods

A pilot study with quasi-experimental pre- and post-test survey design was used to explore student perceptions of the explicit incorporation of ChatGPT version 3.5 into an assignment. The survey was designed by the research team and consisted of three parts: demographic details, 17 dichotomous/Likert-scale questions, and three open-ended questions informed by current literature (see Appendix A). Identical surveys, including the open-ended questions, were offered at the beginning of the course prior to discussions of the assignment and at the end of the semester after submission of the assignment for grading.

Participants

All fourth-year nursing students (n = 152) enrolled in a Western Canadian university for the 2023 fall semester were invited to participate in the study. Convenience sampling supported maximum participation during the data collection period. These participants were targeted as they were the only students in the university enrolled in a nursing course that incorporated GPT-3.5 into an assignment.

Recruitment

A recruitment poster approved by the Health Research Ethics Board (HREB) was shared with eligible students through the course’s online learning platform and was not sent to any student individually. The poster contained a link to the anonymous survey and was shared at the beginning of each data collection period; thus, students were free to access the survey at any time in that month outside of classroom time. The recruitment poster outlined that participation was optional and that there were no consequences to students for not participating. The recruitment poster contained the contact information of the neutral third-party researcher for any questions or clarification. The surveys were not discussed in class, and students were made aware that the surveys were not required for any aspect of the course content, assessments, or grading.

Data Collection Procedures

The authors contacted all instructors for the fall 2023 offering of the course in August and obtained permission for the neutral third-party researcher to share recruitment materials with students in their section. The pre-test survey was open from September 14 to October 13, 2023, and the post-test survey was open December 1 to 24, 2023.

Ethical Considerations

Institutional ethics approval (Ethics Protocol Number 23-0394) was obtained prior to commencing the surveys. As two of the researchers (LN and AW) were actively teaching the students in the same course during the semester in which data collection took place, a researcher (CF) not employed by the institution and with no relation to the students served as a neutral third-party to recruit participants as well as to collect, de-identify, and collate the data via a secure survey platform. The neutral third-party researcher was also available to answer student questions and to reiterate that participation would not influence their academic progress. The students were aware that the two educators were also the researchers and that they would not discuss the surveys with the students.

The surveys were separate from classroom activities and were not part of any content, assignments, or grading for the term. The neutral third-party was the only researcher who had access to the secure survey platform and the raw data. They opened and closed both surveys as well as shared the de-identified data with the educator–researchers after final grades were submitted. The students were made aware that no data sharing would occur until after the final grades were submitted. This strategy was employed to mitigate student perceptions that the surveys were connected with their academic evaluation. Students who chose not to participate faced no consequences, and the educator–researchers were never aware of which students did or did not participate.

Data Analysis

We used descriptive statistics to analyze the demographic characteristics of all respondents—specifically frequencies, range, and mean values (see Table 1). In addition to providing foundational clarity for analysis, these descriptive statistics supported qualitative thematic analysis in the participant subjective responses.

Table 1

Demographics of the Student Respondents

Demographics of the Student Respondents

-> See the list of tables

To assess for statistical significance in perceptions of ChatGPT before and after completing the assignment, pre- and post-test scores were paired and analyzed using IBM Statistical Product and Service Solutions (SPSS) version 29. Frequencies for nominal and ordinal data were generated, comparing pre- and post-test percentages for each question (see Table 2). Additionally, Chi-square tests, a suitable statistical test for nominal and ordinal data to determine statistical significance, were used to compare pre- and post-test data (p < 0.05) (see Table 3).

Table 2

Pre- and Post-test Quantitative Questions (Yes/No)

Pre- and Post-test Quantitative Questions (Yes/No)

-> See the list of tables

Table 3

Pre- and Post-test Likert-Scale Questions

Pre- and Post-test Likert-Scale Questions

Table 3 (continuation)

Pre- and Post-test Likert-Scale Questions

Table 3 (continuation)

Pre- and Post-test Likert-Scale Questions

-> See the list of tables

To examine the responses to the open-ended questions, we employed thematic analysis to look for patterns of meaning (themes) within the subjective text provided by the participants (Braun & Clarke, 2021). The analyzed open-ended questions included: You have indicated that ChatGPT has helped you overcome language or learning disability barriers; can you please explain? Are there any other challenges that you find ChatGPT helps you with? Is there anything else you would like to add? The researchers followed Braun and Clarke’s (2021) six phases and convened after independently reviewing the first 10 responses to ensure analysis was congruent between the researchers. As we became familiar with the data and generated initial codes, we looked for themes together. Finally, we agreed that the themes outlined in this report were an accurate reflection of our interpretation of the participants’ responses. We used direct quotes to illustrate themes. Due to the small number of paired responses (that is, responses from participants who completed both the baseline and the end-of-semester surveys), we used all responses, not exclusively the paired responses, for thematic analysis. We analyzed the baseline and end-of-semester surveys separately to explore any differences over the semester. Further, we viewed this initial process as “a starting point for [a] journey, not a map,” to understanding how students may perceive the use of ChatGPT in a written assignment (Braun et al., 2019, p. 424).

Results

Demographics

In total, 27 responses were paired, and we used these responses for quantitative analysis to answer the main research question: How do nursing students perceive the use of ChatGPT before and after completing an assignment with a GPT-3.5 component? The demographics of this group constitute the core data that could be used to answer the original research data. Ages ranged from 20 to 45. This group self-identified as 44.5% European, 3.7% Indigenous, 3.7% African, 18.5% Asian, and 7.4% Middle Eastern, with 22.2% choosing the “other” category. A total of 96.3% identified as women, and 3.7% identified as men. Demographics for the overall pre-test survey (n = 82) and post-test survey (n = 38) responses were similar (see Table 1). We include all demographic data (the paired responses, the pre-test responses, and the post-test responses) as we make further comparisons to help us understand the data.

Survey Findings: Quantifiable Responses

Analysis of the 27 paired responses finds statistical significance for only one question: Have you used AI/machine learning tools such as ChatGPT in assignments before? During data collection, a surge in the broader university discussion context contributed to some confusion as students received multiple questionnaires or notices from various university entities; however, meaningful information can be gleaned from the overall data set. Most notably, almost half of overall respondents (both pre- and post-test) reported that they believed, to some extent, that using ChatGPT for assignments is not ethical despite significant class discussion and opportunities to ask questions. A total of 67% of all students reported that they had not used ChatGPT for academic assignments prior to the current semester (see Table 2). Many students agreed that ChatGPT helped them overcome language or learning disability barriers (29% paired; 37% overall). At the end of the semester, approximately half agreed that ChatGPT will improve nursing education (59% paired; 58% overall), though fewer agreed that ChatGPT will prepare them for nursing practice (37% paired; 50% overall). As well, 77% of the paired responses (66.3% overall) believed that some aspect of nursing will be replaced by AI tools in their lifetime, with 77% (76% overall) also agreeing that AI tools will become part of their nursing practice (see Table 3).

Survey Findings: Narrative Response Themes

Analysis of the baseline survey (n = 82) revealed a surprising number of students who had never used ChatGPT (67.1%). Those with ChatGPT experience described employing it for “translating,” “using it like a thesaurus,” or using it to “proofread for spelling mistakes” as common applications. Many students expressed skepticism of ChatGPT, stating variations of the idea that “students should be able to write a paper independently.” In addition, many reported that ChatGPT “provides answers that are not applicable” and “references that are not real.” Students expressed much concern of being accused of plagiarism both in the survey and during class time, even though ChatGPT guidance was explicitly included with the assignment description in the course syllabus. More than half of the pre-test respondents left the open-ended questions blank.

While fewer students responded to the end-of-semester survey (n = 38), the responses were more robust. Thematic analysis highlighted three nested themes: “still skeptical,” “just tools,” and “an equalizer.”

“Still Skeptical”

As aligned with quantitative findings, many students still expressed skepticism even after using ChatGPT in an assignment. The major concern continued to be the fear of unintentional plagiarism and/or being accused of plagiarism or cheating. As one student commented, “I do not think we should be incorporating AI into assignments as I believe it dampens creativity and increases the risk of cheating.” Students commented on wanting further direction regarding the implications, explaining “it is in the nursing school’s … whole best interest to learn how to work with this technology, assess students differently and teach students to use a useful resource that is not going anywhere.” One student expressed concern regarding human reliance on such AI tools for “ideas and brainstorming” and the impact in practice environments if future colleagues are “not prepared for complex medical situations.”

“Just Tools”

After completing the assignment, students described ChatGPT and related technologies as “just tools” perceived to meet a wide range of student needs, such as to assist them when they were “overwhelmed by too much information” and needed “a place to start,” to expand thinking to otherwise “unknown” perspectives, and to allow them to see what “is possible” when considering their topic situated within nursing practice. In addition to assistance with basic grammar, spelling, and writing composition, students noted that ChatGPT gave them “feedback on [their] writing.” As well, students reported ChatGPT was “helpful” for “simplifying academic language and medical jargon.” Some students indicated that ChatGPT provided them with an avenue for “conversation” to explore topics, thus supporting their understanding of “complex” and/or “long readings,” which may be particularly meaningful in a cohort who experienced disruptions to their education related to the COVID-19 pandemic. As one student summarized, “ChatGPT has helped me exponentially expand my critical thinking.”

“An Equalizer”

An unexpected number of responses described ChatGPT as “an equalizer.” Students reported that ChatGPT supported their English skills and helped them overcome disability-related barriers that had historically affected their academic success. As one student described, “English is my second language so using [ChatGPT] to help me explain and explore my ideas really helps.” Another student concurred, stating, “expressing myself [in English] properly in essays has always been difficult for me to do.” Students who stated that they had learning disabilities described the perceived benefits of using ChatGPT in conjunction with their academic work. One student stated, “It helped me feel more free and less bound by my disability to learn, communicate … and engage with the world around me.” Students who identified as having attention deficit hyperactivity disorder (ADHD) added that ChatGPT helped with “assignments that [were] hard to start due to ADHD,” helped them “figure out an outline,” and pointed them “in the direction of good sources of information.” Students who stated that they had anxiety reported that using ChatGPT “help[ed them] start” when they were “too overwhelmed to think” and stopped “the circles of overcomplicating [themselves].”

Discussion

Contrary to debates in the higher education literature and media, our study finds a significant number of respondents reported that they did not engage in unauthorized use of ChatGPT for academic work. Further, the students appeared as skeptical of and concerned about academic integrity and cheating as educators. Students are struggling, just as nurse educators are, to figure out how such technologies fit within their learning and future practice. In addition to the importance of providing clear guidance in syllabi and classroom discussions, our findings point to students’ desire to understand ethical use of AI tools with a concurrent need for mentorship and guidance to navigate uncharted territory as AI tools become an inextricable aspect of contemporary education. Student perspectives also need to be honoured and included in policy work currently under way in most academic environments. Unexamined assumptions about student experiences and opinions in relation to AI tools may both underestimate the integrity of student populations and overly limit necessary safe opportunities to explore the parameters of using AI tools that will be embedded in the professional practice environments they enter upon graduation.

The detailed descriptions of AI tools as an “equalizer,” especially considering significant educator efforts, highlight how students continue to experience barriers and inequities in their education. These students described ChatGPT as an enabler to overcome challenges and contribute to their academic success. AI tools might provide an opportunity to learn without being exhausted from the mental power required to regularly deal with barriers. Such inclusive pedagogy is an important aspect of Universal Design for Learning (UDL) (Fornauf & Erickson, 2019). UDL is an evidence-informed approach to education that aims to remove barriers and inequities to learning while supporting students to meaningfully access knowledge to enhance their academic experiences (Davies et al., 2013). In this way, AI tools can be harnessed to elevate educational experiences to be truly inclusive and accessible with as much focus on using such tools for pedagogical good as on plagiarism and cheating (Kumar, 2023).

The integration of ChatGPT and related AI tools into both academic settings and practice environments seems inevitable (Castonguay et al., 2023; Oermann, 2024). Our findings demonstrate that such tools may support students with writing and critical thinking skills; however, demystifying these tools with clear expectations is imperative. As technology and new knowledge changes rapidly, expectations cannot be static. Instead, ongoing conversation about ethical and acceptable use for nursing students and nurse educators is crucial. As these technologies evolve and change, so, too, will nursing, across the career trajectory.

Limitations

Limitations include the need for a convenience sample; findings that were based on self-reported data in which students could perceive power imbalances; the surveys’ being created by the authors and not validated; and the influence of unanticipated multiple concurrent institutional surveys of the same topic. Finally, the frequent use of single-group pre- and post-test design projects for nursing education is a limitation in itself (Spurlock, 2018). Thus, we see this pilot study as a map for future research and do not make any claims of causation. Our original purpose was to create space to talk about AI tools at a time when universities and colleagues were banning such technologies. In less than a year, these bans have been reconsidered, highlighting the temporal limitations of reporting in a climate of intense and rapid change.

Pedagogical Implications

This pilot study offers an opportunity to explore how nursing students perceive AI tools such as ChatGPT in relation to their learning and highlights the profound influence these tools will have on the learning processes of both nursing students and educators. This study provides some insights into how nursing students’ perceptions can support educators to design assignments that balance the use of AI tools with critical thinking skills. Future research aimed to inform best practices for using AI tools in nursing education without compromising academic integrity is greatly needed. We must move towards pedagogical methods that encourage creative and independent thinking while upholding principles of academic integrity.

In addition, better understanding the potential of AI tools to support students in the post-traditional learning environments of today’s classrooms will be essential as we strive for equity. We must be especially mindful that such technologies are poised to be an integral part of health care, as they are increasingly incorporated into practice supports such as decision-making tools. Nursing students’ understandings of how AI tools fit into their education will inform how they practise in digitally enhanced work environments. Evaluating and investigating the long-term impacts of AI tools on learning outcomes, professional development, and patient outcomes begin in the classroom. Nurse educators are at the forefront of this unpredictable workplace transition that is amplifying the digital integration into nursing practice, and generating evidence to support this transition is imperative.

Conclusion

Our pilot study serves as a starting point to understand how we might engage nursing students in a pedagogically sound manner to support their journey to becoming critically thinking professionals in a way that recognizes and works with the changing nature of technologies rather than resists it. As outlined in the Canadian Nurses Association and the Canadian Nursing Informatics Association’s (2024) position statement Nursing Practice in Digitally Enabled Care Environments, these emerging technologies are reshaping the health care landscape. Nurse educators will also need to transform the classroom. The integration of AI tools into undergraduate nursing education can support the competencies required for the nurses of tomorrow. We must continue to explore how ever-evolving generative AI technologies will inevitably change us, as well.