Oral Exams as Alternative (and Authentic) Assessments

Article by Amy J. & James E. Kabrhel, Ph.D., Associate Professors of Chemistry

In the Summer of 2020 during the heat of the COVID-19 pandemic, we learned that we would be allowed to come back to face-to-face instruction in Fall 2020 as long as we used methods allowing social distancing and flexibility for student attendance. I knew I wanted to return to face-to-face teaching as much as possible, but I also understood that many students might contract COVID and be unable to attend in-person class sessions, thereby potentially missing exams or other assessments. I considered several different assessment options to replace the exams I traditionally gave in my CHEM 211 & CHEM 212 (Principles of Chemistry I & II) courses. Some options I considered were online exams and take-home exams. Each has its own pros and cons. For instance, students can easily cheat on online exams thanks to websites like Chegg. The same goes for take-home exams. After carefully considering the pros and cons of various assessment methods, I decided to try oral exams. 

I scheduled the oral exams for each week following the conclusion of each unit. This led to four oral exams per student throughout the semester and one oral final exam per student during finals week. The in-semester exams were 30 min each and the final exam was 60 min each. I googled how to create an Appointment Group in Canvas and created several 30-min timeslots for students to choose from at times that worked for me but hopefully also worked for my students. I tried to offer a variety of days and times throughout each exam week, and I made the Appointment Group available one week before each exam week. I also provided students with instructions on how to select a timeslot in Canvas. The students then went into the Canvas Calendar and chose the timeslot that worked best for them on a first-come-first-served basis. Canvas emails instructors each time an appointment is selected, so I would then create a Zoom link for that oral exam session and send it to the student. I chose to use unique Zoom links for each student to ensure privacy for each exam. 

The student and I logged into Zoom at the scheduled exam time. I had several questions prepared to ask them that I also shared in Zoom for added accessibility. In essence, we had a conversation about the chemistry they had learned for the past month. I was able to give them immediate feedback on their answers and explanations, allowing them to correct their thinking on the spot. I had a grading rubric with me to keep track of how many times I needed to help them answer a question. Each bit of help was a loss of a point (see example assessment table). If a student was stuck on a question, we could move on to the next and return to any left at the end. In most cases, students finished in less than 30 min. In some cases, however, students needed more time, and I emailed them the questions to complete as a take-home exam.

Overal Understanding 20 pts
Ch. 13 30 pts
Ch. 14 30 pts
Prompting 20 pts
TOTAL  100 pts

I did wonder if students would share exam questions with classmates taking the exam later in the week, so I made slight changes to each question for each day of the exam week. For example: 

Which______  diffuses through air most ______ and why? 
Tuesday:  Halogen  quickly 
Wednesday: Halogen  slowly 
Thursday: Noble Gas  quickly 
Friday: Noble Gas  slowly 

The students and I were obviously nervous each time we would meet on Zoom, but after a few minutes, we would ease into the setting. Students became much more adept at explaining their chemical knowledge to me, and by the final exam, they seemed much less nervous and much more comfortable. Student evaluations confirm this observation as students stated they felt their oral communications skills improved throughout the semester. Overall, I think the students may not have loved the oral exams, but they appreciated their flexibility and immediate feedback. Due to this success, my husband and fellow instructor, James Kabrhel, decided to incorporate some oral assessments into his CHEM 302 & 303 (Organic Chemistry I & II) courses, and his impressions are given in the following paragraph.

Organic Chemistry originating from Manitowoc/Sheboygan has been taught through the point-to-point (P2P) modality since the 2000s, but with the addition of the Marinette and Green Bay campuses to the class, providing exams and finding proctors is a much more complicated problem. One solution to the problem was to shift all exams to the take-home format, but as previously mentioned, take-home exams have inherent risks. To balance those risks, an oral assignment and an oral final exam have been added to the course to provide multiple assessment modes for the students. Students must complete an oral assignment in the middle of the semester as a practice with the format, so they are then somewhat comfortable when the final exam comes around. The oral assignment also acts as a mid-semester check-in with the students to see how they are coping with organic chemistry and their classes overall. The addition of the oral exams has been successful enough for me to consider adding an oral part to every exam, not just the final. – J. Kabrhel, personal interview, 2023

You may be thinking that giving oral exams is way too much of a time commitment. I thought it would be as well, but it was not as bad as I expected. I found that it took me the same amount of time to write an oral exam as it did a traditional exam. The difference in time came when comparing the time to administer the exam versus the time to grade the exam (see below). A traditional exam takes one class session to administer but takes much longer to grade, which is dependent on the number of students in the course as well as the difficulty in grading each question. However, administering the oral exams took 30 min per student (9 hours for me as I had 18 students in Fall 2020) but took me almost no time at all to grade because I was grading them while administering them, so the only extra time needed was to type those scores into Canvas. In the end, I found oral exams to be slightly less time intensive than traditional exams.

We (Amy & James) have found the following pros and cons of using oral exams in our courses:

Pros  Cons 
Assessment as a conversation. Big time commitment during exam weeks.
Opportunity for 2nd chances on questions. Not easy to assess complex problem-solving questions.
Misunderstandings corrected immediately. Technology issues.
Immediate feedback; faster grading.
Greater connection with students.  
Improvement in oral communication skills.  

As mentioned in the list above, one con is the huge time commitment during exam weeks. I had to block off nearly my entire week for oral exams. It also was not possible to ask complex problem-solving questions as it simply took students too long to answer them during the exams. Finally, there were some technical issues once in a while. Many of my students live in rural areas where their internet connections are not strong, so we would lose our connection, which took time away from the exams. (This is often when I would have to resort to giving the exam as a take-home, which was not my preference but was a suitable backup option.)

Overall, we feel the pros outweigh the cons, and oral exams are an excellent assessment option if they work for your pedagogical style. They do work best in smaller classes. We feel the max number of students for this method to be manageable would be 24. Beyond that, you would need to have a co-instructor or teaching assistant to help you complete all the exams within one week. In case you are curious, I (Amy) did not continue using oral exams after we returned to fully face-to-face courses, but this is mostly due to my introverted personality 🙂. James, however, is planning on adding more oral assessments to his course due to their equitable nature and the way it allows him to better connect with his students, especially those at campuses he cannot visit regularly. Oral exams are a valid and valuable assessment method, and if you have any questions for us about using oral exams in your courses, we would be more than happy to chat with you about them

Self-Service Tech Help – Using the UWGB IT Knowledgebase

Have you ever encountered a tech issue and wanted to troubleshoot it yourself, but weren’t sure where to begin? For instance, say you and another instructor want to collaborate on course content and share Canvas materials with one another, but you’re unsure of the best way to do this. While you can email DLE or the CATL inbox for advice, you might prefer a quicker solution. Luckily, you can find an immediate answer to your Canvas question by using the UW-Green Bay IT Knowledgebase (UKnowIT). 

If that example doesn’t speak to you, perhaps this one will. Let’s say you are an instructor and one of your students emails you asking for an extension on an assignment due date. You know that Canvas has the capability to do this, and you’ve asked DLE in the past for help with this task. This time you want to try and do it yourself, and you turn to the UWGB IT Knowledgebase (UKnowIT) for guidance. It provides you with a quick answer and a step-by-step guide on how to extend the due date for an individual student on an assignment. What other helpful information is stored away in this knowledgebase? 

The UWGB IT Knowledgebase includes many short tech guides to help UWGB faculty and students on topics from Office 365 to Zoom operations. You can also find step-by-step instructions, complete with screenshots, for common Canvas issues (e.g., exporting grades, combining course sections, course dates/access, etc.). As an instructor, you can even link and share UKnowIT guides in your Canvas course to provide additional technology support for your students (e.g., understanding Canvas modules and organization, embedding files in a Canvas discussion, viewing instructor feedback in Canvas, etc.). 

Searching the IT Knowledgebase (UKnowIT) by typing “Canvas” and filtering through search results.

The UKnowIT is a great resource for UWGB instructors, students, and staff! Grow your confidence in troubleshooting common technology problems by consulting the knowledgebase to discover quick and easy solutions. When it comes to immediate Canvas help beyond what is available on the IT Knowledgebase (UKnowIT), there is even more support for instructors available like the Canvas 24/7 support line and a whole suite of step-by-step guides for each Canvas tool and setting.   

The following frequently searched topics and guides can be found on the UWGB IT Knowledgebase (UKnowIT): 

 

Fostering Information Literacy to Create a More Positive Learning Environment

In an ever-evolving digital landscape that includes unsubstantiated claims on social media, sneaky advertising disguised as unbiased resources, and articles with completely fabricated citations authored by AI-powered chatbots, it is more challenging than ever before to filter out misinformation (and disinformation) from the truth. Like all of us, students can be susceptible to this misinformation, which can have devastating consequences.

How, then, can we as educators work to counteract the misinformation our students are continually exposed to and instead cultivate their information literacy and critical thinking? In the past couple years CATL has sponsored several events around this subject, in which instructors have shared their own experiences, challenges, and triumphs, including a panel at the 2021 Common CAHSS conference and a follow-up event at the 2022 IDI. While we encourage you to watch the recording of the panel in its entirety, we wish to recap some of the main takeaways from those discussions that you may find useful for your own teaching.

Using a Proactive Approach to Information Literacy

Preventative solutions are often the simplest and most effective, and that seems to hold true when it comes to teaching information literacy as well. While the follow-up to this post will focus on directly addressing misinformation that students share during class, this first post will outline measures you can take to reduce the occurrence of such incidents by proactively equipping students with the knowledge and skills they need to engage in your content area.

Teach Thoughtful Critical Analysis

Instructors typically spend a lot of time carefully curating the resources that they include in their courses. As subject matter experts, they know what a “good” source in their field looks like and can select ones that come from reputable sources and are substantiated with solid evidence. Students on the other hand may not have had the chance to hone these critical analysis skills and can get easily led astray when sifting through resources. Fortunately, we can help students develop this important life skill by teaching them how to analyze a source’s credibility.

One framework you could try using with your students is the CRAAP test (or the CAARP test, if you prefer), which besides having a funny, easy to remember acronym, asks students to evaluate a source based its currency, relevance, authority, accuracy, and purpose. Another framework is SOCC (source, observe, contextualize, corroborate) which was developed for history courses but can be applied across disciplines. Frameworks like these help students break down the difficult task of evaluating a resource into smaller steps.

Lateral reading is another excellent strategy to teach students, which is when one conducts external research about the author, publisher, and/or claim to evaluate a source. Often a quick search online can help students identify potentially misleading or outright false information without the need for a more comprehensive analysis. The UWGB Libraries have a plethora of excellent sources on lateral reading and other strategies for evaluating sources, including pre-built lesson ideas. You can even invite a librarian to your class to help conduct a lesson on one of these topics.

Identify Gaps in Students’ Knowledge

Students will likely be walking into your class with varying degrees of background knowledge on the content. In order to help even the playing field, you might want to consider offering students the opportunity to ask questions anonymously, such as in the form of a Canvas survey or a paper “exit ticket.” Once you’ve collected their responses, you can address their concerns in class without identifying the contributors. This helps take the pressure off a student that might be embarrassed to admit their lack of knowledge or misconceptions in front of the whole class.

Another idea is to create an icebreaker activity where students brainstorm common misconceptions about the subject area in small groups. Then, as a class, you can compile their lists and discuss and dispel these myths. Walk through the inquiry process with each misconception—where did this idea come from? Who created it? Why might they have created it? Are there societal or historical contexts that shaped this idea?

In addition to the ideas that students submit, you may wish to do your own inventory of common tropes, myths, stereotypes and misconceptions around the content in your course. If you don’t know the origins of these misconceptions, do a little research so that you can explain their background to students. These types of discussions and activities help students see that challenging their preconceived notions is a natural part of the learning process.

Scaffold Discussions on Complex Topics

Complex issues, such as climate change or structural racism, can be challenging discussion topics due to the substantial number of misconceptions people have. To make things more difficult, students may hold strong opinions on these issues even if their actual understanding of the topic is rather shallow.

To have productive conversations, we first need to teach students the fundamental background knowledge, skills, and frameworks for discussing these topics. Going back to the example of climate change, if students have already had the opportunity to learn about concepts behind climate change, like how matter and energy move through the environment or how humans can influence those movements, they will have an easier time understanding climate change itself once that topic is introduced. Consider ordering the activities in your course to naturally build from foundational concepts towards more complex topics.

Do You Have Other Ideas?

Information literacy is not an easy skill to teach but is so important that we continue to do so in an age when information and misinformation of all kinds is just a Google search away. What are some additional strategies you have used to proactively address common misconceptions or teach information literacy? Or, if you have used any of the methods mentioned here, how did they play out in your own course? Let us know by posting a comment below or by emailing CATL@uwgb.edu. Let’s continue the conversation!

Our special thanks go out to Preston Cherry, Christin DePouw, Lisa Lamson, J P Leary, Brian Merkel, Valerie Murrenus Pilmaier, and Jessica Warwick for their contributions to the 2021 Common CAHSS panel and follow-up 2022 IDI session that served as the inspiration for this article!

What is ChatGPT? Exploring AI Tools and Their Relationship with Education

Artificial intelligence and machine-generated content applications have become a hot topic of discussion in education, from headline news articles in Inside Higher Ed, to UW-Green Bay workshops specific to AI tools. With all this buzz, university instructors and staff may still have lingering questions surrounding what exactly AI applications like ChatGPT are and why they are such a hot topic of discussion in teaching and learning spheres. This CATL blog post will provide readers with definitions for some of these AI tools and explore their possible implications for higher education.  

What is All the Fuss About? 

The most popular AI-text generative tool is ChatGPT by Open AI. What exactly can AI tools like ChatGPT do?  Well, they can write essays and poems, “converse” about the meaning of life, or quickly define terms or even summarize a book. They all offer large potential in their use cases, yet they still come with their own set of limitations, such as producing wrong or false answers. They also tend to be very formal and lack the ability to understand sarcasm, analogies, jokes, and satire, and they are limited by their current datasets which in the case of ChatGPT can only fetch data prior to the year 2021. Below is a more detailed description of some of the most commonly used AI tools. Test them out and ask the AI some questions!  Before you do, be aware that ChatGPT is sometimes at capacity due to high traffic, and that although the tools are currently free, they may require you to establish an account with an email and cell phone number

ChatGPT by Open AI
Perplexity
YouChat

ChatGPT stands for Generative Pre-trained Transformer and is a natural language processing tool powered by AI. That means it generates information in a more conversational style, learns from those conversations, and then can produce even more and more uniquely tailored responses. 

  • Potential use cases: conversational communication, in-depth responses to questions or prompts, creates code, writes essays, answers math problems and shows its work, generates syllabi, formulates application letters for specific job ads, and more.  

Perplexity AI is a chat tool powered by Open AI’s GPT-3 language model that acts as a powerful search engine. When answering a question, the model scours the internet to give an answer and displays the source from which it found the information from.  

  • Potential use cases: answer FAQs, find step-by-step instructions, define terms, and more.  

YouChat is a ChatGPT-like AI search assistant, similar to Perplexity, that acts as a search engine which provides answers and cites sources when asked questions.  

AI tools are not new to education or mainstream news, so what makes tools like ChatGPT the talk of the town, bringing concerns from those connected to education? Put simply, ChatGPT and its contemporaries can understand text and spoken words similar to how human beings can. These tools have become more conversational and corrective, making it difficult to discern originality between what is generated by the AI and what is produced by a human. In addition, the data and algorithms they draw from imitate the way humans learn and can even learn itself by gradually improving its accuracy the more you interact with it. The possibilities of tools like ChatGPT seem to be almost endless — writing complete essays, creating poetry, summarizing books and large texts, creating games, and translating languages and data. This potentially raises questions about the nature of tasks that will be completed by machines in the future and what that means for our learning outcomes for students. Some experts are also discussing to what extent it becomes part of the educational enterprise to teach students how to write effective prompts and use tools like ChatGPT to produce the highest quality, most sophisticated work products reflecting human-machine interaction. 

Instructors and constituents in higher education will need to eventually come to terms with their relationship with these technologies. One way to approach the conversation surrounding AI technology is to consider these applications as tools that educators can choose either to work with or without in their classes. Some may also consider it a part of education to teach their students how to use them most effectively. With any teaching tool we look to incorporate, we must provide proper thought, scaffolding, and framing around what it can do and where it falls short so that students can use the tool responsibly. 

How can I learn more?  

CATL has curated a list of readings and additional resources about AI in education. We will maintain and update this resource regularly as more research on AI-generative tools emerges. Another way to learn more is by staying up to date with the conversations occurring on campus by registering for AI events that will occur during the spring 2023 semester. In addition to these workshops on AI, CATL will continue to provide more blog resources on these tools throughout the spring semester and CATL will be releasing a Canvas course with more resources soon.  

Keep the Conversation Going!  

We want to hear from you! Have you incorporated AI-generative tools in your course instruction? If so, what ideas, challenges, and feedback can you share with us as other instructors consider these tools? What guidelines, syllabus statements or lessons have you added to your course relevant to AI use? What benefits or shortcomings of these new tools have you identified from an instructional standpoint?  

To share your ideas and thoughts please email us at catl@uwgb.edu!  The more we all familiarize ourselves with the tools and engage with them, the clearer the implications the tools will have on teaching and learning.  

Reading and Resources About AI in Education

To help you as you research and explore AI tools, we have provided a list of resources and additional readings on the topic of AI-text generative technology below.  

Higher Education Responses

Playing Around with AI

Additional Reading

Other Center Resources

How can I learn more?  

We will update this blog post as new research emerges on AI. Keep in mind as you review the resources that AI tools and applications will continue to be released in the coming months and develop very rapidly. You may want to check back to this site for updates. Another way to learn more is by staying up to date with the conversations occurring on campus by registering for AI events that will occur during the spring 2023 semester. In addition to these workshops on AI, CATL will continue to provide more blog resources on these tools throughout the spring semester and CATL will be releasing a Canvas course with more resources soon. 

Keep the Conversation Going!  

We want to hear from you! Have you incorporated AI-generative tools in your course instruction? If so, what ideas, challenges, and feedback can you share with us as other instructors consider these tools? What guidelines, syllabus statements or lessons have you added to your course relevant to AI use? What benefits or shortcomings of these new tools have you identified from an instructional standpoint?  

To share your ideas and thoughts please email us at catl@uwgb.edu!  The more we all familiarize ourselves with the tools and engage with them, the clearer the implications the tools will have on teaching and learning.