Strategies for Creating “Generative AI-Resistant” Assessments

The use of generative-AI tools in education has recently garnered significant attention, placing educators in a unique position to consider their roles in higher education and how students engage with such tools. In a previous blog post, we introduced AI technologies and their endless capabilities, as well as potential implications for higher education. Additionally, we provided advice on considerations, precautions, and ethical concerns for using generative-AI in the classroom.

While some educators are excited about integrating AI collaboration into their teaching practices, others are apprehensive about its potential misuse by students. To address these concerns, this blog post presents assignment strategies that can be more “generative AI-resistant.” There are no “AI-proof” assessments, but these suggestions should serve as starting points for creating authentic assignments and/or ones that require demonstration of original and critical thinking.

Assessment Strategies

Be specific and personal

Consider creating assessments that involve tasks ChatGPT struggles to do such as referring to personal anecdotes or student reflections, referencing current events and recent field developments, or administering interviews or making references to specific course materials. Asking students to connect specific details from course materials (readings, lectures, experiments, etc.) to their personal lives or career paths can also help students see relevance in their course activities and engage with them on a deeper level.

Go beyond research papers

Instead of asking students to create a research paper that ChatGPT could do for them, ask students to create an annotated reference list that demonstrates their ability to apply proper research methods and analysis of resources collected. Alternatively, consider asking students to write a paper analyzing a case study that you’ve created yourself.

Mix up the medium

Incorporate assignments that make use of multimedia content such as creating, writing, recording, and producing a podcast episode relevant to course content and ideas.

Flip your classroom

Consider using in-class time for activities like classroom debates and/or peer-to-peer feedback on projects. Grade those efforts that are happening in real-time and under your observation (without generative AI). Note that a flipped classroom approach also allows students to practice higher order thinking and application of the content they’ve learned through homework.

Look at grading

Consider reviewing your course grading criteria and use growth-oriented rubrics that prioritize process over product.

Ask students to show their process

Consider adding assignment elements that ask students to think about the process of their work. This could be done by requiring students to submit notes they took on sources to prepare for their papers or presentations. You could also ask students to show their work in progress as they move toward a final draft (e.g., require submission of a project outline or proposal, annotated bibliography, and multiple drafts).

Allow for growth and resubmission

Consider adding in some flexibility when students fall short of an objective by allowing for revisions or resubmissions on certain assignments. This can reduce the “high stakes” nature of assessments and associated pressures.

Make adjustments to current assessments

Review your existing assignments to see if there are areas where you can have students demonstrate their holistic growth and development. For exams, you could add supplemental reflection questions or even consider adopting oral exams.

A Note on Academic Honesty

It can be easy or tempting to spend a lot of effort trying to catch instances of academic dishonesty using tools like ChatGPT. Although there are detection tools available, such as Turnitin, the effectiveness of AI detection reports remains uncertain due to insufficient data. Please review our blog on Considerations for Using Generative AI Tools to learn more and remember how important it is to communicate explicitly with students about if, when, and how they may use AI in your class.

How Can I Learn More?

CATL has curated a list of readings and additional resources about AI in education! If you have questions, concerns, or ideas specific to generative-AI tools in education and the classroom, please email us at catl@uwgb.edu or set up a consultation!

 

 

Considerations for Using Generative AI Tools

Staying updated on the rapidly evolving generative-AI tools can be challenging, and educators may hold divergent (and strong!) views about them. In a previous blog post, we introduced generative-AI technologies, their capabilities, and implications for higher education. While some educators are enthusiastic about incorporating AI into their teaching methodologies, others may harbor doubts, apprehensions, or simply lack interest in exploring these tools. Regardless of one’s stance, understanding the disruptive impact of these technologies is crucial as we grapple with their ethical and pedagogical implications as educators.

In this post, we will explore some considerations for using generative-AI tools in the classroom, including preliminary precautions and ethical concerns. The more we understand these technologies, the better we can adapt to maximize their potential benefits while minimizing their negative impact.

Things to Consider When Using AI-Powered Tools in your Courses

Understand the inevitability of advancing AI technology.

AI, like many other recent technologies (e.g., personal computers or the internet), will continue to advance and not go away. In fact, they will progress and become better than previous models. This is not something we can “outrun.”

Encourage dialogue on the impact of AI in education

Consider discussing AI technology and its implications with your department, colleagues, and students. In what ways will generative-AI tools change the nature of learning outcomes and even careers in your discipline? How are other instructors responding? In what ways can instructors support each other as they each grapple with these questions?

Provide clear communication with your students on expectations

Whichever camp or situation you may fall into, it is always important to provide students with clear expectations for their use of AI in the classroom. Be specific in your syllabi and assignment descriptions about where and when you will allow or prohibit the use of these tools. You should also make sure whatever guidance you provide is also consistent with UWS Chapter 14 and the communications from our Provost Office. For example statements, view our Syllabus Snippets related to generative-AI

Use generative-AI tools with caution

Exercise caution when using generative-AI tools because the information provided by them may not always be accurate. AI creators, like OpenAI, are upfront about the fact that ChatGPT’s answers aren’t always correct. Due to their ability to hallucinate facts and resources, it’s best to avoid using these tool as a primary source. Be sure to also watch out for potential bias that can appear in outputs by these tools as they are trained on human-generated data.

Offer alternatives for privacy-minded students

If you are asking students to complete an assignment using generative-AI technology, you will also want to provide an opt-out or alternative assignment because students may legitimately not want to provide personal information to sign-up and use certain AI technologies. Many tools openly state they will sell that information.

AI detection tools are not perfect

When using Turnitin’s AI writing detection indicator, it is important to note that there is currently insufficient data to validate its effectiveness. Therefore, results from such reports should be treated as signals that additional review may be necessary. If you suspect academic misconduct, be prepared to support the claim with additional information beyond the detection tool’s report.

Consider ethical and legal issues when using AI tools

As instructors, it is also important to consider the potential ethical, legal, and security risks of AI technologies. Many generative-AI tools are “trained” on the data we put into them, so we must exercise caution when providing prompts to the tools. For example, never put students’ personal information into an AI-powered tool, as this may violate FERPA. Asking students to submit their work (or doing it yourself) to get feedback from ChatGPT or a similar resource puts their intellectual property into the public domain. This should not be done without their explicit consent.

Prepare students to use AI effectively

If you assign tasks that require students to use AI technology, it is important to provide clear instructions about how to do so and not assume students already know. Consider incorporating a discussion on the benefits, limitations, cautions, and ethics of using generative-AI. This could be a valuable in-class activity.

Don’t get caught up in the smoke

Although the capabilities of generative-AI can be scary or worrying at this point, it is best to not get bogged down in the negatives of AI or focus on how to detect cheating through AI use. Are you worried about what AI tools mean for your course materials? Schedule a consultation with us. CATL is here to help!

How Can I Learn More?

CATL has curated a list of readings and additional resources about AI in education, as well as a blog post that explores strategies for creating more “AI-resistant” assessments. If you have questions, concerns, or ideas specific to AI-text generative tools in education and the classroom, please email us at catl@uwgb.edu or set up a consultation!

Exams, Alternative Assessments, and the Question of Proctoring

As we dig into the second half of the spring semester, instructors may now be looking at final assessments for the end of the term. During this time, instructors have many different options when they plan out and assign assessments. In this blog post, we’ll be looking at some alternative options for more traditional proctored exams which instructors can incorporate into their courses. 

The purpose of this post isn’t to say that instructors cannot offer traditional quizzes or tests within their courses. Far from it, in fact. Instead, we are offering an alternative to help avoid over-use of quizzes and exams in line with Palloff and Pratt (2013) where the authors state that, “instructors shouldn’t completely avoid the use of tests and quizzes. These assessments can be appropriate but require instructors to be mindful about when and where they use them.” In this vein, below are several suggestions on how to still incorporate quizzes and exams within a course using certain formats or settings within Canvas that create impactful assessments without a reliance on proctoring.  

Alternatives to Proctoring Traditional Exams: Canvas Settings 

For quizzes and tests which contain multiple choice or other auto-graded questions in Canvas, there are several settings instructors can enable to help encourage academic integrity. First, within the settings of a Canvas Classic Quiz, instructors can set answers to be shuffled between quizzes so that each student sees the answer choices in a different order. Selecting Quiz due dates and setting a time limit on a quiz where students must complete the assessment within the given time are also settings which might be of interest. These options are all in the Classic Quiz settings within Canvas.  

Canvas Classic Quizzes Settings showing shuffle questions, time limit, and quiz attempt settings.

You can also create Classic Quiz question banks and then use question groups to pull questions from one or more question banks. With a question group, you can pull all questions from a bank or set a specific number of questions from the question bank to be randomly selected for the question group. Using a question group to randomize questions within a Canvas Quiz can help deter academic dishonesty.  

Another option in Canvas is to have multiple versions of the same quiz, similar to how you might have a test form A, B, and C, for a paper test in a face-to-face course. Use Canvas to set up multiple versions of an exam or quiz, put students into groups, and then assign each group a different version of the assessment. The directions here discuss assigning an individual student to a quiz; however, you can follow the same directions to assign a quiz to a student group instead.  

Alternatives to Proctoring Traditional Exams: Test Formatting 

Besides selecting specific quiz settings in Canvas which can help to discourage academic dishonesty, instructors can also adjust the format of a quiz or test. One option is to allow students to use open notes combined with a specific time limit while taking a quiz or test. Alternatively, the use of open notes can help prioritize question types such as short-answer or essay questions. These question types focus more on application and tend to encourage more honest and original answers from students than multiple-choice and other auto-graded question types. For example, you might have students conduct an analysis of a case study using key concepts introduced in class or explaining how to solve a specific equation. Often, asking students to explain something from their point of view or discuss how they would approach an example case study are questions that are harder to look up in notes or online.  

Another test format you might consider is to ask students to complete an oral exam. UWGB’s own Dr. Amy Kabrhel and Dr. James Kabrhel recently created a blog post discussing their use of oral exams in place of traditional exams for use in virtual classrooms and other remote learning modalities. 

Alternative Assessments Beyond Traditional Exams

For instructors who may wish to incorporate formative or summative assessments that do not follow a quiz or exam structure, we have a summary of a few alternative options. Popular suggestions for such assessments tend to promote group work, peer review, or other collaborative endeavors. Assessments incorporating such activities tend to foster higher order thinking in students and encourage metacognition, personal reflection on learning, and stimulate more active learning.  

The University of North Dakota Teaching Transformation and Development Academy (TTaDA)  and the Charlotte University Center for Teaching and Learning provide some concrete suggestions of specific types of skills-based assessments that transcend proctoring. Some highlights include portfolios where students select examples of their work over the duration of the course to revisit, analyze, and update to submit for a final assessment. This provides students with the opportunity to portray an increased understanding of course materials, as well as showcase specific pieces of work they found interesting or are proud of.  

Another option instead of assigning quizzes and tests is to allow students to create detailed “study guides” for a hypothetical quiz or test, or questions they believe should be used on a quiz or exam based off the materials covered in class. These activities allow students to show how well they understand the topics and concepts covered in class, while also providing instructors with informal feedback about what information students are identifying as important.  

A different suggestion for alternative assessments in STEM courses in particular came from UND TTaDA where they encourage the use of virtual labs. They highlight an open education resource (OER) created by Merlot University showcasing a collection of virtual labs focused on science, engineering, mathematics, and technology disciplines.  

A final tool instructors can use to look at potential alternative assessments is an interactive Reimagine Assessments resource developed by Emory University’s Center for Faculty Development and Excellence. This tool lets instructors see example activities for alternative assessments based on 4 different assessment goals: content mastery, skill development, analysis, and theory. 

Each of these examples have one common theme, and that is that assessments, either traditional quizzes and tests or alternative assessments, should be designed to not only assess a student’s comfort and mastery of specific knowledge covered within a course, but should also aim to help students develop and hone a variety of professional skills. These skills should both aid students within the classroom and also be applicable in the world beyond higher education. Such skills can include but are not limited to information management, project management, time management, individual and group oral presentation skills, collaboration skills, and the potential to practice various media production and editing skills.  

Assessment Wrap Up

The benefits of being very deliberate in the form and function of an assessment are twofold. First, utilizing different Canvas settings, quiz and test formats, or alternative assessment strategies decreases the dependence of instructors on proctoring. In recent years, proctoring software has become a more controversial topic within higher ed, and the ability to utilize in-person proctoring is equally complicated by various factors, the least of which was the recent COVID pandemic. The second benefit is that reassessing and being critical about when, how, and in what form to present formative and summative assessments can help encourage academic honesty amongst students by not only gauging the level of mastery students have reached throughout a course, but also helping students to develop a skill tool set they can use going forward in higher ed and in future careers. 

If you have any questions or ideas about quizzes, tests, or alternative assessments, please reach out to CATL and schedule a consultation. 

What is ChatGPT? Exploring AI Tools and Their Relationship with Education

Artificial intelligence and machine-generated content applications have become a hot topic of discussion in education, from headline news articles in Inside Higher Ed, to UW-Green Bay workshops specific to AI tools. With all this buzz, university instructors and staff may still have lingering questions surrounding what exactly AI applications like ChatGPT are and why they are such a hot topic of discussion in teaching and learning spheres. This CATL blog post will provide readers with definitions for some of these AI tools and explore their possible implications for higher education.  

What is All the Fuss About? 

The most popular AI-text generative tool is ChatGPT by Open AI. What exactly can AI tools like ChatGPT do?  Well, they can write essays and poems, “converse” about the meaning of life, or quickly define terms or even summarize a book. They all offer large potential in their use cases, yet they still come with their own set of limitations, such as producing wrong or false answers. They also tend to be very formal and lack the ability to understand sarcasm, analogies, jokes, and satire, and they are limited by their current datasets which in the case of ChatGPT can only fetch data prior to the year 2021. Below is a more detailed description of some of the most commonly used AI tools. Test them out and ask the AI some questions!  Before you do, be aware that ChatGPT is sometimes at capacity due to high traffic, and that although the tools are currently free, they may require you to establish an account with an email and cell phone number

ChatGPT by Open AI
Perplexity
YouChat

ChatGPT stands for Generative Pre-trained Transformer and is a natural language processing tool powered by AI. That means it generates information in a more conversational style, learns from those conversations, and then can produce even more and more uniquely tailored responses. 

  • Potential use cases: conversational communication, in-depth responses to questions or prompts, creates code, writes essays, answers math problems and shows its work, generates syllabi, formulates application letters for specific job ads, and more 

Perplexity AI is a chat tool powered by Open AI’s GPT-3 language model that acts as a powerful search engine. When answering a question, the model scours the internet to give an answer and displays the source from which it found the information from.  

  • Potential use cases: answer FAQs, find step-by-step instructions, define terms, and more.  

YouChat is a ChatGPT-like AI search assistant, similar to Perplexity, that acts as a search engine which provides answers and cites sources when asked questions.  

AI tools are not new to education or mainstream news, so what makes tools like ChatGPT the talk of the town, bringing concerns from those connected to education? Put simply, ChatGPT and its contemporaries can understand text and spoken words similar to how human beings can. These tools have become more conversational and corrective, making it difficult to discern originality between what is generated by the AI and what is produced by a human. In addition, the data and algorithms they draw from imitate the way humans learn and can even learn itself by gradually improving its accuracy the more you interact with it. The possibilities of tools like ChatGPT seem to be almost endless — writing complete essays, creating poetry, summarizing books and large texts, creating games, and translating languages and data. This potentially raises questions about the nature of tasks that will be completed by machines in the future and what that means for our learning outcomes for students. Some experts are also discussing to what extent it becomes part of the educational enterprise to teach students how to write effective prompts and use tools like ChatGPT to produce the highest quality, most sophisticated work products reflecting human-machine interaction. 

Instructors and constituents in higher education will need to eventually come to terms with their relationship with these technologies. One way to approach the conversation surrounding AI technology is to consider these applications as tools that educators can choose either to work with or without in their classes. Some may also consider it a part of education to teach their students how to use them most effectively. With any teaching tool we look to incorporate, we must provide proper thought, scaffolding, and framing around what it can do and where it falls short so that students can use the tool responsibly. 

How can I learn more?  

CATL has curated a list of readings and additional resources about AI in education. We will maintain and update this resource regularly as more research on AI-generative tools emerges. In addition, we have more CATL made resources on the topic of AI in this list below:

Keep the Conversation Going!  

We want to hear from you! Have you incorporated AI-generative tools in your course instruction? If so, what ideas, challenges, and feedback can you share with us as other instructors consider these tools? What guidelines, syllabus statements or lessons have you added to your course relevant to AI use? What benefits or shortcomings of these new tools have you identified from an instructional standpoint?  

To share your ideas and thoughts please email us at catl@uwgb.edu!  The more we all familiarize ourselves with the tools and engage with them, the clearer the implications the tools will have on teaching and learning.  

Reading and Resources About AI in Education

To help you as you research and explore AI tools, we have provided a list of resources and additional readings on the topic of Generative AI technology below.

Generative-AI In the Classroom

ChatGPT, built on the GPT-4 system, and other Generative AI platforms, offer unique opportunities for instructors and students to leverage the technology while still providing robust, comprehensive learning experiences. However, some instructors are apprehensive about its potential misuse by learning activities. Below you will find a variety of resources on how to use generative AI in classroom activities, with examples of activities that may not require any usage of AI.

Incorporating Generative-AI

Working Around Generative-AI

Additional Resources on Assessment and Generative-AI

Learning to Use AI Yourself

Playing Around with AI

Additional Commentary on AI (Articles, Podcast, etc.)

Other Center Resources

How Can I Learn More?

We will update this Generative-AI resources blog post as new research emerges on artificial intelligence. Keep in mind as you review the resources that AI tools and applications will continue to be released in the coming months and develop very rapidly. You may want to check back to this site for updates. In addition, we have more CATL made resources on the topic of AI in this list below:

If you have questions, concerns, or ideas specific to AI-text generative tools in education and the classroom, please email us at catl@uwgb.edu or set up a consultation!