What to Do with Student Evaluation Feedback

The student evaluation of teaching process can produce anxiety in instructors at all career stages. No matter how confident you are in your teaching, being evaluated can feel as though your teaching style, pedagogy, and even personality are being put under a microscope. Turning this anxiety into a productive process for your class can be as simple as reframing how you look at results and what you do with the information provided. This blog post will provide guidance about some best practices for before, during, and after the administration of student evaluations of teaching.

Before and During the Administration of Evaluations

In order to successfully utilize the other suggestions provided in this article, you need to be sure you frame student evaluations effectively with your students. According to Ballantyne, Borthwick, and Packer (2000), students are ambivalent about the usefulness of teaching feedback, but research shows that they believe using feedback for the improvement of teaching is the best possible outcome for completing Student Evaluations of Teaching.  This finding provides a useful place to start in framing your discussion with your students. Several weeks before student evaluations of teaching will be sent out by the university, begin discussing the process with your students. Let them know how they will be evaluating you. Will you provide time in class and leave the room? Will they complete the evaluations on their own time? Will you provide all students with an incentive (e.g., an extra credit point) if the class achieves a certain response rate threshold?

[Note: Check out this other CATL post for more specific ways to boost your student response rates. ]

After you have primed your students to complete the feedback form, inform them how this information will be used. Getting a high rate of thoughtful responses is a goal, and this will likely not happen unless students understand that these responses are important to you and that they will be used to improve your teaching practice. No one wants to complete a task that has no bearing on anything. Students may be led to believe that their feedback on the end of the course evaluation doesn’t hold any weight, so let them know why they should complete it. If you review their comments and make changes to your course, let them know that. If you believe evaluation is important to your career, let them know that. Give your students a reason to want to complete the form. It is important that you are clear that the student evaluation of teaching is important to you. Students are unlikely to find value in an evaluation if you do not.

[Note: Because of the time frame of end-of-semester student evaluations, the feedback they provide can’t be used to improve your current class. This is a reason that doing your own mid-semester evaluation is a great idea, so that you can make course corrections where it seems necessary. See this other blog post for some ideas about how to collect mid-semester feedback.]

Once You Receive Feedback

Don’t focus on the negative

It is human nature to read reviews and focus on one negative comment, even when you have 100 positive comments. You might even find yourself trying to guess at the identity of those outliers who are concerned about one portion of your class or one comment you made, even when the rest of the responses are overwhelmingly positive. This isn’t to say that you should not pay attention to negative comments, but don’t let them define your performance in a class. Don’t overcorrect, especially if you don’t have significant evidence to show that the issue is really a problem. Be aware of the potential for instructor identity-related bias in student ratings, as well (Buser, Batz-Barbarich, & Kearns Hayter, 2022). Give yourself time to think and process through negative comments before deciding what really is the best way to proceed.

Set your standards

Before you look at your feedback, define for yourself what strong performance looks like. For example,  if most of your feedback returns as “somewhat agree” or “strongly agree” on the Likert scale items, you may preemptively decide that would be a positive result. Review the feedback you are given and look for trends. Reflect honestly on your practices if you see areas that come up as consistent areas of weakness. While you cannot please everyone, it is important to note where trends occur that show a potential area for growth.

It’s also important to know that students and faculty often don’t perceive excellence in teaching in the same way. (Spooren et al, 2013). Students may focus on factors that are irrelevant to effective teaching and instead base their evaluations on perceived workload, ability to achieve an “easy A” or any number of factors that don’t actually impact teaching and learning.  For this reason, faculty may ignore feedback as “meaningless,” when it could actually be beneficial to their teaching practice. Be conscious of this potential reasoning trap and think through your feedback, and what you may be able to change.

Don’t try to go it alone

Taking your feedback and meeting with a trusted colleague can be a helpful and beneficial activity. Colleagues, especially those who are more seasoned, will often have experienced many of the issues you are having. If you are disappointed in specific results, talk through your teaching process with colleagues and identify potential areas for growth, or discuss what might have caused the feedback you are unhappy with. Reflecting and talking through these scenarios can help you see clearly what you might want to fix or how to ensure better feedback and a better experience for your students.

Commit to using the feedback

“Student evaluation of teaching (SET) only becomes an effective tool for improving teaching and learning when the relevant stakeholders seriously consider and plan appropriate actions according to student feedback” (Wong & Moni, 2014, p. 397). This passage accurately demonstrates the need for something to happen with feedback once it is collected. It is easy and possibly natural to assume that feedback collected and provided after the end of the semester has limited usefulness. While that feedback cannot provide a better experience for the students who shared it, it can be used to improve your instruction going forward. Consider the input from your students, and be conscious of where that feedback leads you, even if it’s in a different direction for your course. It is not a sign of weakness to accept feedback and make changes. It is the natural progression of an educator devoted to excellence.

Recognize there is an institutional context to evaluation

Although it is natural to focus on course evaluations as a process that occurs between you and your students, the reality is that these tools are used in decision-making, and you are not the only one with access to the results. The standard UWGB course evaluation is used for most classes, and not only you, but administrators such as your Chair and the Associate Provost, have access to the results. Those summaries are used in tenure, promotion, and contract renewal decisions, and UW System also requires gathering student feedback on teaching in our courses (see p. 106 of the Faculty Handbook for policy).

What does all this mean? Well, first, student evaluations are clearly valued at UWGB and beyond. They are not, however, the only means for documenting and improving teaching and learning. The Faculty Handbook also points to other sources, such as peer observation, samples of syllabi and assessments, self-reflection, and engagement in professional development (p. 93). Assemble that information in a teaching portfolio, regularly reflect on your own teaching, invite peer feedback, and participate in educational development opportunities (such as CATL workshops!). Second, since you know others will have access to results, talk to them! Seek out your chair, talk through the evaluations, and learn more about how your unit tends to use them administratively. Finally, remember in the end this should all be about enhancing student learning. If the standard UWGB evaluation does not give you the information you feel you need to improve your course and the student experience, you can always administer a supplemental evaluation on your own. And if you need assistance with any of these issues, remember that CATL is here to help!

Scaffolding for Online Learning

As the end of the semester approaches and you begin to review the curricular structure of your courses in the near future, you may recognize the need for more robust scaffolding in content design regarding the online modality. Before reviewing and modifying your course in this capacity, it is important to know what scaffolding is, and why it is important for student learning. Scaffolding, as EdGlossary defines it in education, refers to ‘a variety of instructional techniques used to move students progressively toward stronger understanding and, ultimately, greater independence in the learning process’. Ultimately, the goal of scaffolding is to give students building blocks of learning that lead to better retention and acquisition of knowledge.

The most common place to start with scaffolding that can provide a significant impact is in larger assignments or assessments. A good ‘rule of thumb’ is to begin with the tasks that take a significant portion of time and energy. Breaking an assessment into smaller subtasks creates natural checkpoints for the students to gauge their understanding. This also allows you as the teacher to gain insight into how their knowledge acquisition is going and allows you to slightly alter course if the learning is not going as first imagined – check out CATL’s blog post on ‘small teaching’ for more information on that topic.

For example, if you are requiring students to ultimately create a final essay project, you could create a scaffolded or sequenced set of checkpoints to build towards the final assignment’s conclusion. The University of Michigan’s Center for Writing has a comprehensive breakdown of this sequencing:

  1. Pre-Writing: including proposals, work-in-progress presentations, and research summaries
  2. Writing: including counterarguments, notes, and drafts
  3. Revision: including peer reviews, conferences, and revision plans

The introduction of any of these concepts in an online environment requires intentionality and planning, while ensuring the students remain highly engaged throughout the process. As the students revise their papers, scheduling individual conferences, peer reviews (via online conferences, social annotations via Hypothesis, or via Canvas), and revision plans can all provide beneficial steps for a scaffolded approach to a final essay project. To ensure that the students are understanding what is required of them, be certain that you answer such critical questions as:

  • How are students able to know that they completed the steps required, and how will they know they have completed it satisfactorily?
  • How will you make the connections between the scaffolded activities and the end product clear as students progress systematically through the courses?
  • Have you clearly identified opportunities for students, particularly in the online modality, to get together remotely for feedback, thought-partnering, and/or review?

Another version of scaffolding in the online modality has to do with the structuring of how students gain an understanding of the content. The University of Buffalo’s Office of Curriculum, Assessment, and Teaching Transformation takes the Gradual Release of Responsibility (GRR) model and utilizes it in both a standard classroom, as well as a ‘flipped classroom’ environment. The GRR model focuses on an ‘I Do’, ‘We Do’, ‘You Do’ framework that is very popular in educational scaffolding. This framework for scaffolding could be centered around a larger assignment or exam, but it does not necessarily need to be. The GRR model of scaffolding could also be utilized when breaking down a larger concept for students. See how this model could potentially be utilized in a chemistry lesson surrounding intramolecular forces:

  1. “I Do” – The instructor creates an introductory lesson introducing intramolecular forces, and discusses the types of bonds that atoms can form (ionic, covalent, etc.). The instructor then shows examples of these types of bonds utilizing different atom types via medium of choice.
  2. “We Do” – This portion of the scaffolding could take place between students, working in pairs or small groups identifying the different types of bonds, and providing examples of each. This scaffolding could also include meeting with the instructor, via Teams or Zoom, or through a discussion that provides more of a ‘guided’ approach to the concepts.
  3. “You Do” – Students work on their own to display the learning that they have gathered on the topic. This could be done with a written assignment, discussion board post, low-stake quiz, or any way that the instructor chooses to assess students’ acquisition of knowledge.

These are just a couple of examples how you can integrate scaffolding into your course content for online learning. The critical aspect of scaffolding is purposeful chunking and segmenting of complex concepts and activities for comprehensive knowledge acquisition. It is important to keep in mind that any scaffolding should continue to be aligned to course expectations and learning outcomes as students will be more successful when it is done with consistency in a holistic sense.

If you would like to learn more about how to use scaffolding for online learning in your own course or have examples of how you are already using it, we’d love to hear from you! Feel free to contact the CATL office by email (CATL@uwgb.edu) to let us know where you’ve found success with these strategies, or to schedule a consultation with us.

Exams, Alternative Assessments, and the Question of Proctoring

As we dig into the second half of the spring semester, instructors may now be looking at final assessments for the end of the term. During this time, instructors have many different options when they plan out and assign assessments. In this blog post, we’ll be looking at some alternative options for more traditional proctored exams which instructors can incorporate into their courses. 

The purpose of this post isn’t to say that instructors cannot offer traditional quizzes or tests within their courses. Far from it, in fact. Instead, we are offering an alternative to help avoid over-use of quizzes and exams in line with Palloff and Pratt (2013) where the authors state that, “instructors shouldn’t completely avoid the use of tests and quizzes. These assessments can be appropriate but require instructors to be mindful about when and where they use them.” In this vein, below are several suggestions on how to still incorporate quizzes and exams within a course using certain formats or settings within Canvas that create impactful assessments without a reliance on proctoring.  

Alternatives to Proctoring Traditional Exams: Canvas Settings 

For quizzes and tests which contain multiple choice or other auto-graded questions in Canvas, there are several settings instructors can enable to help encourage academic integrity. First, within the settings of a Canvas Classic Quiz, instructors can set answers to be shuffled between quizzes so that each student sees the answer choices in a different order. Selecting Quiz due dates and setting a time limit on a quiz where students must complete the assessment within the given time are also settings which might be of interest. These options are all in the Classic Quiz settings within Canvas.  

Canvas Classic Quizzes Settings showing shuffle questions, time limit, and quiz attempt settings.

You can also create Classic Quiz question banks and then use question groups to pull questions from one or more question banks. With a question group, you can pull all questions from a bank or set a specific number of questions from the question bank to be randomly selected for the question group. Using a question group to randomize questions within a Canvas Quiz can help deter academic dishonesty.  

Another option in Canvas is to have multiple versions of the same quiz, similar to how you might have a test form A, B, and C, for a paper test in a face-to-face course. Use Canvas to set up multiple versions of an exam or quiz, put students into groups, and then assign each group a different version of the assessment. The directions here discuss assigning an individual student to a quiz; however, you can follow the same directions to assign a quiz to a student group instead.  

Alternatives to Proctoring Traditional Exams: Test Formatting 

Besides selecting specific quiz settings in Canvas which can help to discourage academic dishonesty, instructors can also adjust the format of a quiz or test. One option is to allow students to use open notes combined with a specific time limit while taking a quiz or test. Alternatively, the use of open notes can help prioritize question types such as short-answer or essay questions. These question types focus more on application and tend to encourage more honest and original answers from students than multiple-choice and other auto-graded question types. For example, you might have students conduct an analysis of a case study using key concepts introduced in class or explaining how to solve a specific equation. Often, asking students to explain something from their point of view or discuss how they would approach an example case study are questions that are harder to look up in notes or online.  

Another test format you might consider is to ask students to complete an oral exam. UWGB’s own Dr. Amy Kabrhel and Dr. James Kabrhel recently created a blog post discussing their use of oral exams in place of traditional exams for use in virtual classrooms and other remote learning modalities. 

Alternative Assessments Beyond Traditional Exams

For instructors who may wish to incorporate formative or summative assessments that do not follow a quiz or exam structure, we have a summary of a few alternative options. Popular suggestions for such assessments tend to promote group work, peer review, or other collaborative endeavors. Assessments incorporating such activities tend to foster higher order thinking in students and encourage metacognition, personal reflection on learning, and stimulate more active learning.  

The University of North Dakota Teaching Transformation and Development Academy (TTaDA)  and the Charlotte University Center for Teaching and Learning provide some concrete suggestions of specific types of skills-based assessments that transcend proctoring. Some highlights include portfolios where students select examples of their work over the duration of the course to revisit, analyze, and update to submit for a final assessment. This provides students with the opportunity to portray an increased understanding of course materials, as well as showcase specific pieces of work they found interesting or are proud of.  

Another option instead of assigning quizzes and tests is to allow students to create detailed “study guides” for a hypothetical quiz or test, or questions they believe should be used on a quiz or exam based off the materials covered in class. These activities allow students to show how well they understand the topics and concepts covered in class, while also providing instructors with informal feedback about what information students are identifying as important.  

A different suggestion for alternative assessments in STEM courses in particular came from UND TTaDA where they encourage the use of virtual labs. They highlight an open education resource (OER) created by Merlot University showcasing a collection of virtual labs focused on science, engineering, mathematics, and technology disciplines.  

A final tool instructors can use to look at potential alternative assessments is an interactive Reimagine Assessments resource developed by Emory University’s Center for Faculty Development and Excellence. This tool lets instructors see example activities for alternative assessments based on 4 different assessment goals: content mastery, skill development, analysis, and theory. 

Each of these examples have one common theme, and that is that assessments, either traditional quizzes and tests or alternative assessments, should be designed to not only assess a student’s comfort and mastery of specific knowledge covered within a course, but should also aim to help students develop and hone a variety of professional skills. These skills should both aid students within the classroom and also be applicable in the world beyond higher education. Such skills can include but are not limited to information management, project management, time management, individual and group oral presentation skills, collaboration skills, and the potential to practice various media production and editing skills.  

Assessment Wrap Up

The benefits of being very deliberate in the form and function of an assessment are twofold. First, utilizing different Canvas settings, quiz and test formats, or alternative assessment strategies decreases the dependence of instructors on proctoring. In recent years, proctoring software has become a more controversial topic within higher ed, and the ability to utilize in-person proctoring is equally complicated by various factors, the least of which was the recent COVID pandemic. The second benefit is that reassessing and being critical about when, how, and in what form to present formative and summative assessments can help encourage academic honesty amongst students by not only gauging the level of mastery students have reached throughout a course, but also helping students to develop a skill tool set they can use going forward in higher ed and in future careers. 

If you have any questions or ideas about quizzes, tests, or alternative assessments, please reach out to CATL and schedule a consultation. 

Exam Wrappers

An exam wrapper is a way for students to reflect on their experience on an exam. It is meant for learners to look again at the techniques they use to get ready for an exam, identify strategies they can use to prepare for later assessments, and consider how similar strategies might help them in their studies in and beyond your course.

Sometimes also called “exam debriefs,” these follow-up reflection activities are often called “exam wrappers” because they serve as a wrap-up for the work done on an exam. They’re also meant for students to further articulate context and relevance for what the exam covered—’wrapping’ some additional meaning around the work they’ve done.

Exam wrappers largely attempt to get at (and point students toward reflecting on) the following:

  • The amount of time and effort put into studying
  • Study habits used
  • Whether students engage with course objectives (especially to direct their studying)
  • Reasons students lose or believe they lose points (whether they missed foundational knowledge, made “silly mistakes,” environmental factors and distraction, etc.)
  • Possible interventions or adjustments

How you implement an exam wrapper is up to you. Some possible strategies include:

  • An exam wrapper counting for an improvement of one half letter grade (from BC to B, for example) on the exam.
  • Requiring students complete a wrapper to turn in alongside corrections for full or partial credit on missed questions.
  • Pairing an exam wrapper with instructor- or TA-led review sessions for later exams. Note: If you go this route, it’s still a good idea to have students complete the wrapper activity shortly after receiving feedback on the exam they’re reviewing so it and their study habits are fresh in their minds.
  • Offering the exam wrapper as ‘makeup’ work for one or more formative activities which led up to the exam.
  • Offering course-level extra credit.
  • Some combination of any or all of these!

As students complete the survey/worksheet, encourage them to think about their answers as they go. A few examples:

  • The question about techniques lists good techniques for studying. Could you adopt one or more of these?
  • There is a question about how you use the learning objectives in the course. You might not yet, but doing so is a good way to get to know why we are doing what we are in this course—including why exam questions are what they are.
  • You’re also asked why you think you lost points on the exam. For the more frequent reasons, what adjustments might you make to avoid these in the future? Do you need to study differently or maybe just slow down when taking the exam?

The last few questions in the examples provides below ask students to articulate responses to these sorts of reflections.

If you are interested in trying out an exam wrapper, we might recommend beginning with a basic Canvas Survey. We have a file you can download and import into a Canvas course to get you started.

If you’re looking for something a little more robust or want to do more with the data, you can take a look at an Example Exam Wrapper Assignment using Qualtrics here. If you’d like a copy of the survey used in this example, you can download this Exam_Wrapper QSF File (Click to Download) and import it into your Qualtrics account (click for instructions). (Note: you do not want to re-use the link in the sample assignment since you will not be able to access the data/results.) The same assignment is available here in Word document format (Click to Download) if you prefer. The Canvas Survey version above is also very similar.

Additional examples of exam wrappers for various disciplines can be found on Carnegie Mellon University’s Eberly Center page on Exam Wrappers.

For further reading, see:

  • Badir, A. et al. 2018. “Exam Wrappers, Reflection, and Student Performance in Engineering Mechanics.” 2018 ASEE Annual Conference & Exposition, Salt Lake City, Utah. https://peer.asee.org/30462
  • Gizem Gezer-Templeton, et al. 2017. “Use of Exam Wrappers to Enhance Students’ Metacognitive Skills in a Large Introductory Food Science and Human Nutrition Course.” Research in Food Science Education 16(1): 28-36. https://doi.org/10.1111/1541-4329.12103
  • Pate, A. et al. 2019. “The use of exam wrappers to promote metacognition.” Currents in Pharmacy Teaching and Learning 11(5): 492-8. https://doi.org/10.1016/j.cptl.2019.02.008

Backward Design

What is backward design?

Backward design is a three-stage process for designing a course:

  1. Identify outcomes or the desired results of learning.
  2. Determine what counts as acceptable evidence of learning.
  3. Plan learning experiences or instruction that will lead students. to achieve your outcomes and provide evidence of learning.

Instructional designers call this process “backward” because one starts with the outcomes and works backward towards daily lessons. One strategy for arraying course experiences is called “scaffolding”—building on prior knowledge to reach new knowledge. A detailed overview of scaffolding can be found here.

This process is most closely associated with a book called Understanding by Design by Grant Wiggins and Jay McTighe (2005). You can read a condensed version of the book if you like or watch the video below to learn more.

How does backward design relate to digital democracy?

The structure of your course speaks volumes to students. When learning outcomes, activities, and assessments work together, students know what to do and how their work contributes to their overall success in the course. At a time when face-to-face communication is at a premium, the structure of the course stands stands in for the check-ins at the beginning of class, where the instructor orients students to how the activities of the day contribute to the overall goals of the course. When the course itself makes transparent connections between outcomes, activities, and assessments you and your students do not have to lumber off to a web conference to discuss these matters. Moreover, students who are not able to attend a web conference are not left out.

How can backward design help me?

Backward design ensures that your outcomes, activities, and assessments work together. As William Strunk, Jr. said of writing: “sentences should contain no unnecessary words… for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts” (Elements of Style, Rule 17.) So too should a course contain no unnecessary work. Backward design helps you determine which elements of your course to keep (those that help students meet the course outcomes as measured by the assessments) and which elements may be edited out (those which do not help your students show mastery of the course outcomes).

Alignment

Diagram showing a "triangle" of intended outcomes, teaching and learning activities, and assessment.

Backward design implies a linear method (albeit in reverse) where one starts with outcomes works backward to assessment and then back again to daily learning activities. In an ideal world, that would be true. But in reality it is more useful to think of backward design as a process that instructors can enter at any of the three points and then use the other two to triangulate their course design.

For example, you may have a really great group project that you think will make a great assessment. To ensure the project fits in your course you will want to attach learning outcomes to it and plan daily activities that will lead your students toward being successful on the final project. This group project is every bit as aligned as one which started with the outcomes and proceeded backward to the assessment (group project) and back again to the daily activities. The key idea is that all three sides of the triangle should work together.

Applying Backward Design

Backward design may be useful for updating your course in multiple ways.

First, you will want to see that your assessments provide evidence that your students have mastered your learning outcomes.

Second, as you consider the various ways students may access your class (face-to-face, online, synchronous online, etc.) you will want to see if your daily activities are aligned equally well in each learning environment. For example, if you do a think-pair-share in a face-to-face lecture so that students can practice key concepts, you will want to do a similar activity in the online environment, such as a journal activity where students explain key concepts in their own terms. The activities don’t have to be the same but they should provide an equivalent experience that is equally well aligned with assessments and learning outcomes.

Finally, backward design serves as a way to check if something is worth doing (does it align?) and a method for incorporating new elements that you can’t resist doing (how can it be made to align with the outcomes, assessments, and learning activities?).