Oral Exams as Alternative (and Authentic) Assessments

Article by Amy J. & James E. Kabrhel, Ph.D., Associate Professors of Chemistry

In the Summer of 2020 during the heat of the COVID-19 pandemic, we learned that we would be allowed to come back to face-to-face instruction in Fall 2020 as long as we used methods allowing social distancing and flexibility for student attendance. I knew I wanted to return to face-to-face teaching as much as possible, but I also understood that many students might contract COVID and be unable to attend in-person class sessions, thereby potentially missing exams or other assessments. I considered several different assessment options to replace the exams I traditionally gave in my CHEM 211 & CHEM 212 (Principles of Chemistry I & II) courses. Some options I considered were online exams and take-home exams. Each has its own pros and cons. For instance, students can easily cheat on online exams thanks to websites like Chegg. The same goes for take-home exams. After carefully considering the pros and cons of various assessment methods, I decided to try oral exams. 

I scheduled the oral exams for each week following the conclusion of each unit. This led to four oral exams per student throughout the semester and one oral final exam per student during finals week. The in-semester exams were 30 min each and the final exam was 60 min each. I googled how to create an Appointment Group in Canvas and created several 30-min timeslots for students to choose from at times that worked for me but hopefully also worked for my students. I tried to offer a variety of days and times throughout each exam week, and I made the Appointment Group available one week before each exam week. I also provided students with instructions on how to select a timeslot in Canvas. The students then went into the Canvas Calendar and chose the timeslot that worked best for them on a first-come-first-served basis. Canvas emails instructors each time an appointment is selected, so I would then create a Zoom link for that oral exam session and send it to the student. I chose to use unique Zoom links for each student to ensure privacy for each exam. 

The student and I logged into Zoom at the scheduled exam time. I had several questions prepared to ask them that I also shared in Zoom for added accessibility. In essence, we had a conversation about the chemistry they had learned for the past month. I was able to give them immediate feedback on their answers and explanations, allowing them to correct their thinking on the spot. I had a grading rubric with me to keep track of how many times I needed to help them answer a question. Each bit of help was a loss of a point (see example assessment table). If a student was stuck on a question, we could move on to the next and return to any left at the end. In most cases, students finished in less than 30 min. In some cases, however, students needed more time, and I emailed them the questions to complete as a take-home exam.

Overal Understanding 20 pts
Ch. 13 30 pts
Ch. 14 30 pts
Prompting 20 pts
TOTAL  100 pts

I did wonder if students would share exam questions with classmates taking the exam later in the week, so I made slight changes to each question for each day of the exam week. For example: 

Which______  diffuses through air most ______ and why? 
Tuesday:  Halogen  quickly 
Wednesday: Halogen  slowly 
Thursday: Noble Gas  quickly 
Friday: Noble Gas  slowly 

The students and I were obviously nervous each time we would meet on Zoom, but after a few minutes, we would ease into the setting. Students became much more adept at explaining their chemical knowledge to me, and by the final exam, they seemed much less nervous and much more comfortable. Student evaluations confirm this observation as students stated they felt their oral communications skills improved throughout the semester. Overall, I think the students may not have loved the oral exams, but they appreciated their flexibility and immediate feedback. Due to this success, my husband and fellow instructor, James Kabrhel, decided to incorporate some oral assessments into his CHEM 302 & 303 (Organic Chemistry I & II) courses, and his impressions are given in the following paragraph.

Organic Chemistry originating from Manitowoc/Sheboygan has been taught through the point-to-point (P2P) modality since the 2000s, but with the addition of the Marinette and Green Bay campuses to the class, providing exams and finding proctors is a much more complicated problem. One solution to the problem was to shift all exams to the take-home format, but as previously mentioned, take-home exams have inherent risks. To balance those risks, an oral assignment and an oral final exam have been added to the course to provide multiple assessment modes for the students. Students must complete an oral assignment in the middle of the semester as a practice with the format, so they are then somewhat comfortable when the final exam comes around. The oral assignment also acts as a mid-semester check-in with the students to see how they are coping with organic chemistry and their classes overall. The addition of the oral exams has been successful enough for me to consider adding an oral part to every exam, not just the final. – J. Kabrhel, personal interview, 2023

You may be thinking that giving oral exams is way too much of a time commitment. I thought it would be as well, but it was not as bad as I expected. I found that it took me the same amount of time to write an oral exam as it did a traditional exam. The difference in time came when comparing the time to administer the exam versus the time to grade the exam (see below). A traditional exam takes one class session to administer but takes much longer to grade, which is dependent on the number of students in the course as well as the difficulty in grading each question. However, administering the oral exams took 30 min per student (9 hours for me as I had 18 students in Fall 2020) but took me almost no time at all to grade because I was grading them while administering them, so the only extra time needed was to type those scores into Canvas. In the end, I found oral exams to be slightly less time intensive than traditional exams.

We (Amy & James) have found the following pros and cons of using oral exams in our courses:

Pros  Cons 
Assessment as a conversation. Big time commitment during exam weeks.
Opportunity for 2nd chances on questions. Not easy to assess complex problem-solving questions.
Misunderstandings corrected immediately. Technology issues.
Immediate feedback; faster grading.
Greater connection with students.  
Improvement in oral communication skills.  

As mentioned in the list above, one con is the huge time commitment during exam weeks. I had to block off nearly my entire week for oral exams. It also was not possible to ask complex problem-solving questions as it simply took students too long to answer them during the exams. Finally, there were some technical issues once in a while. Many of my students live in rural areas where their internet connections are not strong, so we would lose our connection, which took time away from the exams. (This is often when I would have to resort to giving the exam as a take-home, which was not my preference but was a suitable backup option.)

Overall, we feel the pros outweigh the cons, and oral exams are an excellent assessment option if they work for your pedagogical style. They do work best in smaller classes. We feel the max number of students for this method to be manageable would be 24. Beyond that, you would need to have a co-instructor or teaching assistant to help you complete all the exams within one week. In case you are curious, I (Amy) did not continue using oral exams after we returned to fully face-to-face courses, but this is mostly due to my introverted personality 🙂. James, however, is planning on adding more oral assessments to his course due to their equitable nature and the way it allows him to better connect with his students, especially those at campuses he cannot visit regularly. Oral exams are a valid and valuable assessment method, and if you have any questions for us about using oral exams in your courses, we would be more than happy to chat with you about them

Self-Service Tech Help – Using the UWGB IT Knowledgebase

Have you ever encountered a tech issue and wanted to troubleshoot it yourself, but weren’t sure where to begin? For instance, say you and another instructor want to collaborate on course content and share Canvas materials with one another, but you’re unsure of the best way to do this. While you can email DLE or the CATL inbox for advice, you might prefer a quicker solution. Luckily, you can find an immediate answer to your Canvas question by using the UW-Green Bay IT Knowledgebase (UKnowIT). 

If that example doesn’t speak to you, perhaps this one will. Let’s say you are an instructor and one of your students emails you asking for an extension on an assignment due date. You know that Canvas has the capability to do this, and you’ve asked DLE in the past for help with this task. This time you want to try and do it yourself, and you turn to the UWGB IT Knowledgebase (UKnowIT) for guidance. It provides you with a quick answer and a step-by-step guide on how to extend the due date for an individual student on an assignment. What other helpful information is stored away in this knowledgebase? 

The UWGB IT Knowledgebase includes many short tech guides to help UWGB faculty and students on topics from Office 365 to Zoom operations. You can also find step-by-step instructions, complete with screenshots, for common Canvas issues (e.g., exporting grades, combining course sections, course dates/access, etc.). As an instructor, you can even link and share UKnowIT guides in your Canvas course to provide additional technology support for your students (e.g., understanding Canvas modules and organization, embedding files in a Canvas discussion, viewing instructor feedback in Canvas, etc.). 

Searching the IT Knowledgebase (UKnowIT) by typing “Canvas” and filtering through search results.

The UKnowIT is a great resource for UWGB instructors, students, and staff! Grow your confidence in troubleshooting common technology problems by consulting the knowledgebase to discover quick and easy solutions. When it comes to immediate Canvas help beyond what is available on the IT Knowledgebase (UKnowIT), there is even more support for instructors available like the Canvas 24/7 support line and a whole suite of step-by-step guides for each Canvas tool and setting.   

The following frequently searched topics and guides can be found on the UWGB IT Knowledgebase (UKnowIT): 

 

Fostering Information Literacy to Create a More Positive Learning Environment

In an ever-evolving digital landscape that includes unsubstantiated claims on social media, sneaky advertising disguised as unbiased resources, and articles with completely fabricated citations authored by AI-powered chatbots, it is more challenging than ever before to filter out misinformation (and disinformation) from the truth. Like all of us, students can be susceptible to this misinformation, which can have devastating consequences.

How, then, can we as educators work to counteract the misinformation our students are continually exposed to and instead cultivate their information literacy and critical thinking? In the past couple years CATL has sponsored several events around this subject, in which instructors have shared their own experiences, challenges, and triumphs, including a panel at the 2021 Common CAHSS conference and a follow-up event at the 2022 IDI. While we encourage you to watch the recording of the panel in its entirety, we wish to recap some of the main takeaways from those discussions that you may find useful for your own teaching.

Using a Proactive Approach to Information Literacy

Preventative solutions are often the simplest and most effective, and that seems to hold true when it comes to teaching information literacy as well. While the follow-up to this post will focus on directly addressing misinformation that students share during class, this first post will outline measures you can take to reduce the occurrence of such incidents by proactively equipping students with the knowledge and skills they need to engage in your content area.

Teach Thoughtful Critical Analysis

Instructors typically spend a lot of time carefully curating the resources that they include in their courses. As subject matter experts, they know what a “good” source in their field looks like and can select ones that come from reputable sources and are substantiated with solid evidence. Students on the other hand may not have had the chance to hone these critical analysis skills and can get easily led astray when sifting through resources. Fortunately, we can help students develop this important life skill by teaching them how to analyze a source’s credibility.

One framework you could try using with your students is the CRAAP test (or the CAARP test, if you prefer), which besides having a funny, easy to remember acronym, asks students to evaluate a source based its currency, relevance, authority, accuracy, and purpose. Another framework is SOCC (source, observe, contextualize, corroborate) which was developed for history courses but can be applied across disciplines. Frameworks like these help students break down the difficult task of evaluating a resource into smaller steps.

Lateral reading is another excellent strategy to teach students, which is when one conducts external research about the author, publisher, and/or claim to evaluate a source. Often a quick search online can help students identify potentially misleading or outright false information without the need for a more comprehensive analysis. The UWGB Libraries have a plethora of excellent sources on lateral reading and other strategies for evaluating sources, including pre-built lesson ideas. You can even invite a librarian to your class to help conduct a lesson on one of these topics.

Identify Gaps in Students’ Knowledge

Students will likely be walking into your class with varying degrees of background knowledge on the content. In order to help even the playing field, you might want to consider offering students the opportunity to ask questions anonymously, such as in the form of a Canvas survey or a paper “exit ticket.” Once you’ve collected their responses, you can address their concerns in class without identifying the contributors. This helps take the pressure off a student that might be embarrassed to admit their lack of knowledge or misconceptions in front of the whole class.

Another idea is to create an icebreaker activity where students brainstorm common misconceptions about the subject area in small groups. Then, as a class, you can compile their lists and discuss and dispel these myths. Walk through the inquiry process with each misconception—where did this idea come from? Who created it? Why might they have created it? Are there societal or historical contexts that shaped this idea?

In addition to the ideas that students submit, you may wish to do your own inventory of common tropes, myths, stereotypes and misconceptions around the content in your course. If you don’t know the origins of these misconceptions, do a little research so that you can explain their background to students. These types of discussions and activities help students see that challenging their preconceived notions is a natural part of the learning process.

Scaffold Discussions on Complex Topics

Complex issues, such as climate change or structural racism, can be challenging discussion topics due to the substantial number of misconceptions people have. To make things more difficult, students may hold strong opinions on these issues even if their actual understanding of the topic is rather shallow.

To have productive conversations, we first need to teach students the fundamental background knowledge, skills, and frameworks for discussing these topics. Going back to the example of climate change, if students have already had the opportunity to learn about concepts behind climate change, like how matter and energy move through the environment or how humans can influence those movements, they will have an easier time understanding climate change itself once that topic is introduced. Consider ordering the activities in your course to naturally build from foundational concepts towards more complex topics.

Do You Have Other Ideas?

Information literacy is not an easy skill to teach but is so important that we continue to do so in an age when information and misinformation of all kinds is just a Google search away. What are some additional strategies you have used to proactively address common misconceptions or teach information literacy? Or, if you have used any of the methods mentioned here, how did they play out in your own course? Let us know by posting a comment below or by emailing CATL@uwgb.edu. Let’s continue the conversation!


Our special thanks go out to Preston Cherry, Christin DePouw, Lisa Lamson, J P Leary, Brian Merkel, Valerie Murrenus Pilmaier, and Jessica Warwick for their contributions to the 2021 Common CAHSS panel and follow-up 2022 IDI session that served as the inspiration for this article!

What is ChatGPT? Exploring AI Tools and Their Relationship with Education

Artificial Intelligence (AI) and machine-generated content have become prominent in educational discussions. Amidst technical jargon and concerns about the impact of traditional writing and learning, understanding these topics can be overwhelming. This toolbox guide simplifies the generative AI landscape, providing clear definitions and insights into some commonly used generative AI tools.

What is Generative AI?

To provide an introduction to generative AI, CATL has created an informative video presentation. This video, paired with interactive PowerPoints slides, serves as a valuable resource for understanding how generative AI tools work, their capabilities, and limitations.

Introduction to Generative AI – CATL Presentation Slides (PDF)

Common Generative AI Tools

One of the most popular AI-powered text generators is ChatGPT by OpenAI. Since its November 2022 release, various companies have developed their own generative AI applications based on or in direct competitive with OpenAI’s framework. Learn more about common generative AI tools below.

Note: For UWGB faculty, staff, and students, we recommend using Microsoft Copilot and other tools that do not require users to provide personal information in the sign-up process.

What Can Generative AI Tools Do?

The generative AI tools we’ve discussed so far are all trained on large datasets that produce outputs based on patterns in that dataset. User prompts and feedback can be used to improve their outputs and models, so these tools are constantly evolving. Explore below to learn about general use cases for generative AI tools and their limitations.

Generative AI tools can be used in a multitude of ways. Some common uses cases for text-based generative AI tools include: 

  • Language generation: Users can ask them to create essays, poems, or code snippets on a given topic.  
  • Information retrieval: Users can ask them to answer simple non-academic questions like “explain the rules of football to me” or “what is the correct way to use a semicolon?”  
  • Language translation: Users can ask them to translate words or phrases into different languages.  
  • Text summarization: Users can ask them to condense notes from a lecture and or long texts, including entire books, into shorter summaries. 
  • Idea generation & editorial assistance: Users can ask them to brainstorm and generate ideas for a story or a research outline or provide feedback on writing to make it more concise or formal.  

However, these tools also have some limitations, including but not limited to:  

  • Lack of real-world understanding: They do not understand the context and/or logic of the real world. They do not understand sarcasm, analogies, jokes, and satire. For example, an output created by the technology may be grammatically correct, but semantically is nonsensical or contradictory.  
  • Dependent upon the data it is trained on: They may produce outputs that are not accurate, relevant, or current because they rely on the data they are trained on. 
  • False results or hallucinated responses: They may produce outputs that are false, misleading, or plagiarized from other sources, and are unable to verify the accuracy of their outputs.  
  • Machine learning bias: They may produce outputs that are discriminatory or harmful due to bias in the data they are trained on.  

The possibilities of tools like ChatGPT seem to be almost endless — writing complete essays, creating poetry, summarizing books and large texts, creating games, and translating languages and data. ChatGPT and its contemporaries can understand text and spoken words similar to how human beings can. These tools have become more conversational and corrective with each update, making it difficult to discern between what is generated by an AI and what is produced by a human. In addition, the data and algorithms they draw from imitate the way humans learn and can gradually improve their accuracy the more you interact with it. As explored above, they offer large potential in their use cases, yet they still come with their own set of limitations to consider.

What Does This Mean for Educators?

The existence of this technology raises questions about which tasks will be completed all or in part by machines in the future and what that means for our learning outcomes and assessments. Some experts are also discussing to what extent it should become part of the educational enterprise to teach students how to write effective AI prompts and use tools like ChatGPT thoughtfully to produce work that balances quality with efficiency.

One way to approach the conversation surrounding AI technology is to consider these applications as tools that educators can choose either to work with or without in their classes. Some may also consider teaching their students how to use these tools most effectively and/or integrating lessons on AI ethics into their teaching. With any teaching tool we look to incorporate, we must provide proper thought, scaffolding, and framing around what it can do and where it falls short so that students can use the tool responsibly.

Learn More

Explore even more CATL resources related to AI in education:

If you have questions, concerns, or ideas specific to generative AI tools in education and the classroom, please email us at catl@uwgb.edu or set up a consultation!

Reading and Resources About AI in Education

To help you as you research and explore AI tools, we have provided a list of resources and additional readings on the topic of Generative AI technology below.

Additionally, CATL developed a GenAI checklist for instructors that will help you assess the extent to which generative AI will affect your courses and provide guidance on steps for moving forward.

Generative Artificial Intelligence In the Classroom

ChatGPT, built on the GPT-4 system, and other Generative AI platforms, offer unique opportunities for instructors and students to leverage the technology while still providing robust, comprehensive learning experiences. However, some instructors are apprehensive about its potential misuse by learning activities. Below you will find a variety of resources on how to use generative AI in classroom activities, with examples of activities that may not require any usage of AI.

Add a Generative AI Syllabus Statement

Incorporating Generative AI

Working Around Generative AI

Additional Resources on Assessment and Generative AI

Learning to Use AI Yourself

Playing Around with AI

Additional Commentary on AI (Articles, Podcast, etc.)

Other Center Resources

Learn More

Explore even more CATL resources related to AI in education:

If you have questions, concerns, or ideas specific to generative AI tools in education and the classroom, please email us at catl@uwgb.edu or set up a consultation!