Raising Student Evaluation Response Rates

Student evaluations of teaching play a crucial role in professional and course development and in the personnel review process. If they are to be useful, it is important that the data they provide be as accurate as possible. Unfortunately, students are not always motivated to complete them, perhaps because they don’t realize their voice is valued in this process. It is also well-documented that response rates for online evaluations are lower than for in-person administration. There are concrete strategies available to increase participation; however, and research points to creating a positive classroom culture and having explicit discussions of evaluations and specific ways they have been/will be used to inform courses as particularly effective (Chapman & Joines, 2017). A summary of some additional techniques is included below.

  • Make an announcement about evaluations in person (if possible) and in your Canvas course. Do this at the beginning and near the end of the survey period. Be sure to explain why student feedback is important and give specific examples of how you have used it in the past to revise classes. If you are teaching online, you could accomplish this with a short video.
  • Provide some time in class or a space online for students to ask questions about evaluations and their uses at UWGB.
  • Allow students time in class to complete their surveys, making sure to leave the “room” when you do. You should not be present when students complete evaluations. If you do provide time, note that ending class early to do so may only result in students leaving. In online courses, you might factor additional time for evaluations into your calculations of workload for the week and let students know that.
  • Assure students that the surveys are anonymous. Reinforce the point by leaving the physical or Zoom room when the students take them.
  • Include the direct link or QR code for your specific course evaluation in the Canvas announcement. You might also attach this helpful Knowledge Base article so students know how to locate the surveys for all their classes or even show in-person students where to find the necessary information.
  • Put “Complete Course Evaluation” as a task in your Canvas shell and include it on the calendar so it shows up on students’ “To Do” list for the class.
  • Bring the topic of the evaluations up several times during the period they are open, so they remain top of mind for students, even if you’ve already allowed time to complete them in class.
  • Monitor overall response rates for your classes during the open period. Ethically, you cannot award credit for completing an evaluation, and remember they are anonymous. You can, though, make classes aware of response rates and even create a contest between course sections to see who can achieve the highest overall response rate by a specific date. Offer a non-tangible prize to the winning class, such as bragging rights or a choice on a final assignment.
  • Throughout the semester, foster an environment of open communication and respect with students, which may motivate them to see their feedback as valued and worth taking the time to provide for you.

Chapman, D.D., & Joines, J.A. (2017). Strategies for increasing response rates for online end-of-course evaluations. International Journal of Teaching and Learning in Higher Education, 29(1), 47-60. http://www.isetl.org/ijtlhe/

Scaffolding for Online Learning

As the end of the semester approaches and you begin to review the curricular structure of your courses in the near future, you may recognize the need for more robust scaffolding in content design regarding the online modality. Before reviewing and modifying your course in this capacity, it is important to know what scaffolding is, and why it is important for student learning. Scaffolding, as EdGlossary defines it in education, refers to ‘a variety of instructional techniques used to move students progressively toward stronger understanding and, ultimately, greater independence in the learning process’. Ultimately, the goal of scaffolding is to give students building blocks of learning that lead to better retention and acquisition of knowledge.

The most common place to start with scaffolding that can provide a significant impact is in larger assignments or assessments. A good ‘rule of thumb’ is to begin with the tasks that take a significant portion of time and energy. Breaking an assessment into smaller subtasks creates natural checkpoints for the students to gauge their understanding. This also allows you as the teacher to gain insight into how their knowledge acquisition is going and allows you to slightly alter course if the learning is not going as first imagined – check out CATL’s blog post on ‘small teaching’ for more information on that topic.

For example, if you are requiring students to ultimately create a final essay project, you could create a scaffolded or sequenced set of checkpoints to build towards the final assignment’s conclusion. The University of Michigan’s Center for Writing has a comprehensive breakdown of this sequencing:

  1. Pre-Writing: including proposals, work-in-progress presentations, and research summaries
  2. Writing: including counterarguments, notes, and drafts
  3. Revision: including peer reviews, conferences, and revision plans

The introduction of any of these concepts in an online environment requires intentionality and planning, while ensuring the students remain highly engaged throughout the process. As the students revise their papers, scheduling individual conferences, peer reviews (via online conferences, social annotations via Hypothesis, or via Canvas), and revision plans can all provide beneficial steps for a scaffolded approach to a final essay project. To ensure that the students are understanding what is required of them, be certain that you answer such critical questions as:

  • How are students able to know that they completed the steps required, and how will they know they have completed it satisfactorily?
  • How will you make the connections between the scaffolded activities and the end product clear as students progress systematically through the courses?
  • Have you clearly identified opportunities for students, particularly in the online modality, to get together remotely for feedback, thought-partnering, and/or review?

Another version of scaffolding in the online modality has to do with the structuring of how students gain an understanding of the content. The University of Buffalo’s Office of Curriculum, Assessment, and Teaching Transformation takes the Gradual Release of Responsibility (GRR) model and utilizes it in both a standard classroom, as well as a ‘flipped classroom’ environment. The GRR model focuses on an ‘I Do’, ‘We Do’, ‘You Do’ framework that is very popular in educational scaffolding. This framework for scaffolding could be centered around a larger assignment or exam, but it does not necessarily need to be. The GRR model of scaffolding could also be utilized when breaking down a larger concept for students. See how this model could potentially be utilized in a chemistry lesson surrounding intramolecular forces:

  1. “I Do” – The instructor creates an introductory lesson introducing intramolecular forces, and discusses the types of bonds that atoms can form (ionic, covalent, etc.). The instructor then shows examples of these types of bonds utilizing different atom types via medium of choice.
  2. “We Do” – This portion of the scaffolding could take place between students, working in pairs or small groups identifying the different types of bonds, and providing examples of each. This scaffolding could also include meeting with the instructor, via Teams or Zoom, or through a discussion that provides more of a ‘guided’ approach to the concepts.
  3. “You Do” – Students work on their own to display the learning that they have gathered on the topic. This could be done with a written assignment, discussion board post, low-stake quiz, or any way that the instructor chooses to assess students’ acquisition of knowledge.

These are just a couple of examples how you can integrate scaffolding into your course content for online learning. The critical aspect of scaffolding is purposeful chunking and segmenting of complex concepts and activities for comprehensive knowledge acquisition. It is important to keep in mind that any scaffolding should continue to be aligned to course expectations and learning outcomes as students will be more successful when it is done with consistency in a holistic sense.

If you would like to learn more about how to use scaffolding for online learning in your own course or have examples of how you are already using it, we’d love to hear from you! Feel free to contact the CATL office by email (CATL@uwgb.edu) to let us know where you’ve found success with these strategies, or to schedule a consultation with us.

A rusted weathervane with an overcast sky in the background

Addressing Misinformation in the Classroom

A few weeks ago, we posted about how to proactively foster information literacy in your classes. Even if you’ve taken many of the precautions we suggested, there will still inevitably come a time when misinformation arises in the classroom. Whether it is in the context of a Canvas discussion board or during an in-person discussion, an obviously false claim from a student can leave you feeling blindsided if you’re not prepared. With a little work, though, you can often turn these interactions into constructive learning opportunities. This post explores steps you can take to intervene when a student shares misinformation in one of your courses.

Ask Clarifying Questions

When a student shares something incorrect, your first instinct may be to shut them down. In order to turn this into a potential learning opportunity, however, it can be useful to allow the student space to work through what they’ve just shared. Consider asking follow-up questions to clarify what the student is saying and to probe their rationale (a method known as Socratic questioning). Without accusing or assuming, you could pose open-ended questions about where they learned this information and what they know about the source or author in terms of expertise or potential biases. These types of reflective questions help students analyze their misconceptions and may lead them to see the flaws in their own claim.

Be Cognizant of Tone and Body Language

As you engage with a student, be aware of your tone and body language (or written tone, in the case of asynchronous communications like discussion boards). Keep calm and take a moment to collect your thoughts, if needed. Once you are ready to address their remark, keep the tone conversational instead of accusatory. It is important that students don’t misconstrue your response as adversarial.

Try to gauge the student’s nonverbal cues during your dialog as well. If they seem hesitant to share once you begin asking them to clarify their claim, it might help to reassure them that complex topics often lead to misconceptions, and that they might not have had a prior chance to learn about this topic in depth. This helps emphasize that it is not a moral failing on the part of the student for believing misinformation. Instead, remind them inquiry and analysis are a natural part of the learning process (as well as a part of our institutional learning outcomes).

Offer an Invitation to Learn More

Depending on your course and the nature of the student’s misconception, your dialog might naturally segue into a side lesson to discuss the topic at hand. For example, if the claim the student made is a common misconception related to your discipline, you could use this as an opportunity to teach why the misconception exists, where it comes from, and how it might be harmful. If you have sources on hand that help illustrate your point, you might highlight how you determined your sources’ credibility and what measures the sources used to reduce potential bias in their findings, such as using a double-blind setup for a scientific study. You could also use this as an opportunity to teach about how to identify potential bias in a source. Often these situations can be a gateway for a healthy discussion about common misconceptions and real-world applications of the course’s content.

Still, it may not always be appropriate to turn a student’s remark into a teachable moment for the whole class. If a student seems particularly defensive or uncomfortable, or the topic seems emotionally charged for them, you can offer to continue the conversation with the student one-on-one after class. This allows you to shift the class’s attention back to the lesson and off of the student and prevent a situation from escalating.

Address Sensitive Issues with Extra Caution

It is worth mentioning that misconceptions can be extremely damaging when they double as microaggressions. A microaggression — or a subtle display of bias or prejudice — perpetuates harmful stereotypes or misconceptions about a group of people. The Eberly Center from Carnegie Mellon University has a quick guide on addressing microaggressions that outlines additional steps you may wish to take to mitigate the situation in addition to the ones outlined above. Another great resource is this guide on identifying and responding to microaggressions, authored by Dr. Kevin Nadal, a professor of psychology and a leading researcher on microaggressions. Responding to microaggressions in the classroom is crucial for maintaining a safe and supportive learning environment, so they should be handled with extra care.

Do You Have Other Ideas?

Handling misinformation can be tricky, but we hope that these suggestions can help you feel a bit more prepared the next time you encounter it in the classroom. How do you address students’ misconceptions in your own classes? Have any tips for turning these opportunities into teachable moments? We invite you to engage in thoughtful dialog on this topic — post a comment below or email CATL@uwgb.edu to continue the conversation!

Our special thanks go out to Preston Cherry, Christin DePouw, Lisa Lamson, J P Leary, Brian Merkel, Valerie Murrenus Pilmaier, and Jessica Warwick for their contributions to the 2021 Common CAHSS panel and follow-up 2022 IDI session that served as the inspiration for this article!

Exams, Alternative Assessments, and the Question of Proctoring

As we dig into the second half of the spring semester, instructors may now be looking at final assessments for the end of the term. During this time, instructors have many different options when they plan out and assign assessments. In this blog post, we’ll be looking at some alternative options for more traditional proctored exams which instructors can incorporate into their courses. 

The purpose of this post isn’t to say that instructors cannot offer traditional quizzes or tests within their courses. Far from it, in fact. Instead, we are offering an alternative to help avoid over-use of quizzes and exams in line with Palloff and Pratt (2013) where the authors state that, “instructors shouldn’t completely avoid the use of tests and quizzes. These assessments can be appropriate but require instructors to be mindful about when and where they use them.” In this vein, below are several suggestions on how to still incorporate quizzes and exams within a course using certain formats or settings within Canvas that create impactful assessments without a reliance on proctoring.  

Alternatives to Proctoring Traditional Exams: Canvas Settings 

For quizzes and tests which contain multiple choice or other auto-graded questions in Canvas, there are several settings instructors can enable to help encourage academic integrity. First, within the settings of a Canvas Classic Quiz, instructors can set answers to be shuffled between quizzes so that each student sees the answer choices in a different order. Selecting Quiz due dates and setting a time limit on a quiz where students must complete the assessment within the given time are also settings which might be of interest. These options are all in the Classic Quiz settings within Canvas.  

Canvas Classic Quizzes Settings showing shuffle questions, time limit, and quiz attempt settings.

You can also create Classic Quiz question banks and then use question groups to pull questions from one or more question banks. With a question group, you can pull all questions from a bank or set a specific number of questions from the question bank to be randomly selected for the question group. Using a question group to randomize questions within a Canvas Quiz can help deter academic dishonesty.  

Another option in Canvas is to have multiple versions of the same quiz, similar to how you might have a test form A, B, and C, for a paper test in a face-to-face course. Use Canvas to set up multiple versions of an exam or quiz, put students into groups, and then assign each group a different version of the assessment. The directions here discuss assigning an individual student to a quiz; however, you can follow the same directions to assign a quiz to a student group instead.  

Alternatives to Proctoring Traditional Exams: Test Formatting 

Besides selecting specific quiz settings in Canvas which can help to discourage academic dishonesty, instructors can also adjust the format of a quiz or test. One option is to allow students to use open notes combined with a specific time limit while taking a quiz or test. Alternatively, the use of open notes can help prioritize question types such as short-answer or essay questions. These question types focus more on application and tend to encourage more honest and original answers from students than multiple-choice and other auto-graded question types. For example, you might have students conduct an analysis of a case study using key concepts introduced in class or explaining how to solve a specific equation. Often, asking students to explain something from their point of view or discuss how they would approach an example case study are questions that are harder to look up in notes or online.  

Another test format you might consider is to ask students to complete an oral exam. UWGB’s own Dr. Amy Kabrhel and Dr. James Kabrhel recently created a blog post discussing their use of oral exams in place of traditional exams for use in virtual classrooms and other remote learning modalities. 

Alternative Assessments Beyond Traditional Exams

For instructors who may wish to incorporate formative or summative assessments that do not follow a quiz or exam structure, we have a summary of a few alternative options. Popular suggestions for such assessments tend to promote group work, peer review, or other collaborative endeavors. Assessments incorporating such activities tend to foster higher order thinking in students and encourage metacognition, personal reflection on learning, and stimulate more active learning.  

The University of North Dakota Teaching Transformation and Development Academy (TTaDA)  and the Charlotte University Center for Teaching and Learning provide some concrete suggestions of specific types of skills-based assessments that transcend proctoring. Some highlights include portfolios where students select examples of their work over the duration of the course to revisit, analyze, and update to submit for a final assessment. This provides students with the opportunity to portray an increased understanding of course materials, as well as showcase specific pieces of work they found interesting or are proud of.  

Another option instead of assigning quizzes and tests is to allow students to create detailed “study guides” for a hypothetical quiz or test, or questions they believe should be used on a quiz or exam based off the materials covered in class. These activities allow students to show how well they understand the topics and concepts covered in class, while also providing instructors with informal feedback about what information students are identifying as important.  

A different suggestion for alternative assessments in STEM courses in particular came from UND TTaDA where they encourage the use of virtual labs. They highlight an open education resource (OER) created by Merlot University showcasing a collection of virtual labs focused on science, engineering, mathematics, and technology disciplines.  

A final tool instructors can use to look at potential alternative assessments is an interactive Reimagine Assessments resource developed by Emory University’s Center for Faculty Development and Excellence. This tool lets instructors see example activities for alternative assessments based on 4 different assessment goals: content mastery, skill development, analysis, and theory. 

Each of these examples have one common theme, and that is that assessments, either traditional quizzes and tests or alternative assessments, should be designed to not only assess a student’s comfort and mastery of specific knowledge covered within a course, but should also aim to help students develop and hone a variety of professional skills. These skills should both aid students within the classroom and also be applicable in the world beyond higher education. Such skills can include but are not limited to information management, project management, time management, individual and group oral presentation skills, collaboration skills, and the potential to practice various media production and editing skills.  

Assessment Wrap Up

The benefits of being very deliberate in the form and function of an assessment are twofold. First, utilizing different Canvas settings, quiz and test formats, or alternative assessment strategies decreases the dependence of instructors on proctoring. In recent years, proctoring software has become a more controversial topic within higher ed, and the ability to utilize in-person proctoring is equally complicated by various factors, the least of which was the recent COVID pandemic. The second benefit is that reassessing and being critical about when, how, and in what form to present formative and summative assessments can help encourage academic honesty amongst students by not only gauging the level of mastery students have reached throughout a course, but also helping students to develop a skill tool set they can use going forward in higher ed and in future careers. 

If you have any questions or ideas about quizzes, tests, or alternative assessments, please reach out to CATL and schedule a consultation. 

Oral Exams as Alternative (and Authentic) Assessments

Article by Amy J. & James E. Kabrhel, Ph.D., Associate Professors of Chemistry

In the Summer of 2020 during the heat of the COVID-19 pandemic, we learned that we would be allowed to come back to face-to-face instruction in Fall 2020 as long as we used methods allowing social distancing and flexibility for student attendance. I knew I wanted to return to face-to-face teaching as much as possible, but I also understood that many students might contract COVID and be unable to attend in-person class sessions, thereby potentially missing exams or other assessments. I considered several different assessment options to replace the exams I traditionally gave in my CHEM 211 & CHEM 212 (Principles of Chemistry I & II) courses. Some options I considered were online exams and take-home exams. Each has its own pros and cons. For instance, students can easily cheat on online exams thanks to websites like Chegg. The same goes for take-home exams. After carefully considering the pros and cons of various assessment methods, I decided to try oral exams. 

I scheduled the oral exams for each week following the conclusion of each unit. This led to four oral exams per student throughout the semester and one oral final exam per student during finals week. The in-semester exams were 30 min each and the final exam was 60 min each. I googled how to create an Appointment Group in Canvas and created several 30-min timeslots for students to choose from at times that worked for me but hopefully also worked for my students. I tried to offer a variety of days and times throughout each exam week, and I made the Appointment Group available one week before each exam week. I also provided students with instructions on how to select a timeslot in Canvas. The students then went into the Canvas Calendar and chose the timeslot that worked best for them on a first-come-first-served basis. Canvas emails instructors each time an appointment is selected, so I would then create a Zoom link for that oral exam session and send it to the student. I chose to use unique Zoom links for each student to ensure privacy for each exam. 

The student and I logged into Zoom at the scheduled exam time. I had several questions prepared to ask them that I also shared in Zoom for added accessibility. In essence, we had a conversation about the chemistry they had learned for the past month. I was able to give them immediate feedback on their answers and explanations, allowing them to correct their thinking on the spot. I had a grading rubric with me to keep track of how many times I needed to help them answer a question. Each bit of help was a loss of a point (see example assessment table). If a student was stuck on a question, we could move on to the next and return to any left at the end. In most cases, students finished in less than 30 min. In some cases, however, students needed more time, and I emailed them the questions to complete as a take-home exam.

Overal Understanding 20 pts
Ch. 13 30 pts
Ch. 14 30 pts
Prompting 20 pts
TOTAL  100 pts

I did wonder if students would share exam questions with classmates taking the exam later in the week, so I made slight changes to each question for each day of the exam week. For example: 

Which______  diffuses through air most ______ and why? 
Tuesday:  Halogen  quickly 
Wednesday: Halogen  slowly 
Thursday: Noble Gas  quickly 
Friday: Noble Gas  slowly 

The students and I were obviously nervous each time we would meet on Zoom, but after a few minutes, we would ease into the setting. Students became much more adept at explaining their chemical knowledge to me, and by the final exam, they seemed much less nervous and much more comfortable. Student evaluations confirm this observation as students stated they felt their oral communications skills improved throughout the semester. Overall, I think the students may not have loved the oral exams, but they appreciated their flexibility and immediate feedback. Due to this success, my husband and fellow instructor, James Kabrhel, decided to incorporate some oral assessments into his CHEM 302 & 303 (Organic Chemistry I & II) courses, and his impressions are given in the following paragraph.

Organic Chemistry originating from Manitowoc/Sheboygan has been taught through the point-to-point (P2P) modality since the 2000s, but with the addition of the Marinette and Green Bay campuses to the class, providing exams and finding proctors is a much more complicated problem. One solution to the problem was to shift all exams to the take-home format, but as previously mentioned, take-home exams have inherent risks. To balance those risks, an oral assignment and an oral final exam have been added to the course to provide multiple assessment modes for the students. Students must complete an oral assignment in the middle of the semester as a practice with the format, so they are then somewhat comfortable when the final exam comes around. The oral assignment also acts as a mid-semester check-in with the students to see how they are coping with organic chemistry and their classes overall. The addition of the oral exams has been successful enough for me to consider adding an oral part to every exam, not just the final. – J. Kabrhel, personal interview, 2023

You may be thinking that giving oral exams is way too much of a time commitment. I thought it would be as well, but it was not as bad as I expected. I found that it took me the same amount of time to write an oral exam as it did a traditional exam. The difference in time came when comparing the time to administer the exam versus the time to grade the exam (see below). A traditional exam takes one class session to administer but takes much longer to grade, which is dependent on the number of students in the course as well as the difficulty in grading each question. However, administering the oral exams took 30 min per student (9 hours for me as I had 18 students in Fall 2020) but took me almost no time at all to grade because I was grading them while administering them, so the only extra time needed was to type those scores into Canvas. In the end, I found oral exams to be slightly less time intensive than traditional exams.

We (Amy & James) have found the following pros and cons of using oral exams in our courses:

Pros  Cons 
Assessment as a conversation. Big time commitment during exam weeks.
Opportunity for 2nd chances on questions. Not easy to assess complex problem-solving questions.
Misunderstandings corrected immediately. Technology issues.
Immediate feedback; faster grading.
Greater connection with students.  
Improvement in oral communication skills.  

As mentioned in the list above, one con is the huge time commitment during exam weeks. I had to block off nearly my entire week for oral exams. It also was not possible to ask complex problem-solving questions as it simply took students too long to answer them during the exams. Finally, there were some technical issues once in a while. Many of my students live in rural areas where their internet connections are not strong, so we would lose our connection, which took time away from the exams. (This is often when I would have to resort to giving the exam as a take-home, which was not my preference but was a suitable backup option.)

Overall, we feel the pros outweigh the cons, and oral exams are an excellent assessment option if they work for your pedagogical style. They do work best in smaller classes. We feel the max number of students for this method to be manageable would be 24. Beyond that, you would need to have a co-instructor or teaching assistant to help you complete all the exams within one week. In case you are curious, I (Amy) did not continue using oral exams after we returned to fully face-to-face courses, but this is mostly due to my introverted personality 🙂. James, however, is planning on adding more oral assessments to his course due to their equitable nature and the way it allows him to better connect with his students, especially those at campuses he cannot visit regularly. Oral exams are a valid and valuable assessment method, and if you have any questions for us about using oral exams in your courses, we would be more than happy to chat with you about them