Event Follow-Up: Improving Accessibility in Learning Materials

On Wednesday, Mar. 6, 2024, CATL teamed up with Assistant Professor of Humanities, Kristopher Purzycki, for a workshop on improving the accessibility of educational resources shared in courses and on campus. This session explored common accessibility pitfalls in crafting digital learning materials, covering tasks like creating and sharing PowerPoint presentations, PDFS, and Canvas elements such as media and syllabi. As a continuation of this workshop, we’ve complied practical accessibility tips and demonstrations for instructors to incorporate when creating learning materials.

Prioritizing Accessibility Matters for Student Success

Meeting certain accessibility standards is not just about compliance with the Americans with Disabilities Act; it is also crucial for enhancing student success and engagement. Accessibility (specifically digital accessibility) proactively eliminates barriers during the design and creation phase of materials.

In cases where accessibility measures still pose challenges for learners, students can work with Student Accessibility Services (SAS) to seek formal accommodations and instructors will work with SAS to fulfill the accommodation request. Many students may not disclose their disabilities to their university or face other obstacles hindering them from receiving formal accommodation. Consequently, academic success often relies on students’ individual efforts and faculty commitment to accessible learning materials. While not proposing a complete overhaul of course materials, CATL hopes to promote simple steps to enhance the accessibility of educational learning materials, all in the pursuit of student success.

  • Make course changes based on level of seriousness.
  • Learn and adapt based on experiences and student feedback.
  • Use the UWGB library as a resource to help refresh and update your class materials/readings.
  • Use the Accessibility Checkers available to you in Microsoft Office (like Word, PowerPoint, Excel) and Canvas). 

Canvas Accessibility Tools to Help Review Your Course

Expand the titles below to learn how to use the accessibility tools and checks available to you in Canvas.

How to Use the Canvas Accessibility Checker – Video Demo

Validate Links in Your Canvas Course – Video Demo

Note: This video is demonstration is from Arizona State University Learning Experience (LX) and displays their specific instance of Canvas. While UWGB’s Canvas may operate and look different, the validate course link application works the same. Need more? View the Instructor (Canvas) guide on Validating Links in Canvas.

 

Using the Canvas Course Accessibility Checker UDOIT – Video Overview

Learn even more with UWGB's Knowledgebase guide on using the UDOIT Cloud Accessibility tool to check your Canvas course accessibility.

Video Accessibility with Kaltura My Media and Automatic Closed Captions

Expand the titles below to learn how to upload your own course videos or YouTube finds to Kaltura My Media. This allows for automatic closed captioning, caption editing, and transcription addition for videos in your Canvas courses or those shared with students.

How to Upload Videos and Add Captions with Kaltura My Media – Video Demo 

How to Embed Videos and Add Transcripts with Kaltura My Media – Video Demo

Tip: You can adjust the max embed size of your video under the Embed Settings option. Feel free to use this to adjust the size of your video display in your Canvas course. 

PDF Accessibility with Adobe Acrobat – Optical Character Recognition (OCR) Scanning

Expand the title below to learn how to enhance the accessibility of your PDFs by using OCR  scanning. While OCR scanning doesn’t guarantee full accessibility for assistive technologies like screen readers, Adobe Acrobat Pro offers additional tools to improve accessibility before sharing PDFs digitally.

How to Use OCR Scanning with Adobe Acrobat Pro for PDFs – Video Demo

Tip: Before creating your own PDF documents and PDF scans of readings, contact the UWGB library and ask if they already have a digital resource available.  

Image Accessibility and Informative Alt Text

Expand the title below to learn more about writing helpful alt text for images with specific examples, such as when you are creating your syllabus.

How to Add Alt Text in Microsoft Word and PowerPoint – Video Demo

A Note About Your Syllabus

Your syllabus is a great resource for our students and their first look into your class and learning environment. Because of this, your syllabus should include language that makes your desire for student success obvious. This can be done by incorporating course norms that encourage students to reach out to you if materials are not accessible for them. At UWGB instructors must include an “Accommodation Statement” on their syllabus. While not a requirement, instructors can show their commitment to accessibility and student success by including an additional accessibility statement. See an example of this type of Accessibility from Bates College below.

"Bates College is committed to creating a learning environment that meets the needs of its diverse student body. If you anticipate or experience any barriers to learning in this course, please feel welcome to discuss your concerns with me." – Bates College: Sample Syllabus Accessibility Statement

Learn More

If you’d like to learn more about accessibility, we encourage you to sign up for LITE 120, a self-paced training course that covers the basics of accessibility in Canvas, as well as SAS’s training course on creating accessible documents (i.e., with Word, PowerPoint, or PDF). Plus, check out CATL’s top 10 dos and don’ts of digital accessibility for even more resources.

Related Events and Opportunities

Join us as we conclude this semester’s workshop series with a session on “Using Universal Design for Learning (UDL)to Increase Access” led by the director of UW-Green Bay’s Student Accessibility Services, Lynn Niemi, and Art and Design Professor, Alison Gates. Attendees will continue the conversation about neurodiversity and explore how to use UDL to remove barriers in course materials and increase student access. This workshop will be held virtually via Zoom on Apr. 3rd, 2024, from 3:30 – 4:30 p.m. Registration for the April workshop on UDL is already open.

As always, CATL also welcomes you to connect with us if you’d like to learn more about any of these topics. Send us an email or request a consultation to get started!

 

Event Follow-Up: Students’ Experiences at UWGB via Neurodiverse Viewpoints

On Wednesday, Feb. 7, 2024, CATL collaborated with Assistant Vice Chancellor Stacie Christian to host a student panel on neurodiversity. Six student panelists shared their experiences as neurodiverse learners, including common barriers and misconceptions related to neurodiversity. One of the topics the panel discussed was how instructors can support them. A few common themes emerged from students’ responses so we’ve compiled them below, along with resources for ways you might implement these recommendations in your teaching.

Make Assignment Details Transparent

The student panelists shared that they find it extremely helpful when professors explain the purpose of an assignment and provide clear instructions. Their recommendation aligns with the transparency in learning and teaching (TILT) framework, a concept you may be familiar with if you’ve taken LITE 201. The TILT framework is an evidence-based approach to assignment design in which instructors demystify activities by explaining their purpose, detailing the task that students need to complete, and providing concrete grading criteria. Not sure where to start? Check out this checklist for designing transparent assignments from TILT Higher Ed. Or, for a deeper dive into the topic, consider taking a look at this webinar recording on transparent assignment design.

Explicitly Communicate Your Support

One of the “unwritten rules” of college is that students can go to their instructors when they have a question about the course or the need to connect with another institutional resource, such as tutoring or counseling. While this fact may be obvious to some students, it is not to everyone. Whether due to anxiety, trouble picking up on subtext, or unfamiliarity with the norms of higher education, some students may not ask their instructor for help unless they are given explicit permission to do so. Panelists suggested that instructors include a statement in their syllabus to remind students that they can come to the instructor if they have questions or concerns for help and/or referral to the best resource. It’s a small action but adding a statement like this can help reassure students that you care about their success and wellbeing. For more ideas on how to create a welcoming syllabus, check out this post on liquid syllabi and CATL’s liquid syllabus template. If you want to explore other ways of building trust with your students, consider creating a “getting to know you” survey, establishing class norms, or incorporating a name pronunciation activity.

Provide Alternative Formats for Information

Several student panelists emphasized the importance of providing alternate ways of communicating information whenever possible. This recommendation is not only related to “multiple means of representation” from universal design for learning (UDL) theory, but it also aligns with best practices for digital accessibility. Adding alternative means of representation doesn’t have to be complicated. For example, if you include audio or video files in your course, try to pick resources that also provide captions or a transcript. Or, if you use images, make sure you include a caption or alt text when the image is being used to convey information. If you’d like to learn more about accessibility, we encourage you to sign up for LITE 120, a self-paced training course that covers the basics of accessibility in Canvas, as well as SAS’s training course on creating accessible documents (i.e., with Word, PowerPoint, or PDF).

Related Events and Opportunities

Want to learn more about supporting diverse learners? CATL’s “Workshop Wednesday” series this semester has two upcoming sessions that may be of interest to you! First, on Wednesday, Mar. 6, we’ll take a look at how to make course materials more accessible. Then, on Wednesday, Apr. 3, we’ll explore universal design for learning (UDL) and some practical ways to apply UDL concepts in our teaching and learning. Both workshops will be from 3:30 – 4:30 p.m. via Zoom. Registration for the March workshop on accessibility is already open. Stay tuned for details on registration for April’s workshop.

As always, CATL also welcomes you to connect with us if you’d like to learn more about any of these topics. Send us an email or request a consultation to get started!

“Zeroing in” on Canvas Gradebook Accuracy

A major benefit of using the Canvas gradebook to keep your grades is that it gives students a live and continuously updated view of their standing in the course. For better or worse, students trust that the grade shown to them in Canvas is an accurate measure of their current achievement and a predictor of their final grade. Students use the running total grade shown to them in the Canvas gradebook to set goals for upcoming assignments which will help them achieve their desired final grade. Unfortunately, mistakes and instructor misunderstandings about how Canvas calculates total grades may lead to the total grade students see in a course being misleading or inaccurate, and that can have negative effects on a student’s ability to plan for future coursework. Making sure your Canvas gradebook is accurate and up to date throughout the term also helps prevent final grade “surprises” and grade disputes. The Canvas gradebook practice that most frequently leads to students seeing misleading total grade calculations is leaving missing assignments ungraded. This article explains the importance of regularly entering scores of zero in Canvas for missing work, which is a necessary step for making sure your Canvas gradebook is working for students and not against them.

Because of the way Canvas treats assignments with no grade when calculating a student’s total grade, students who have a missing assignment see a higher total grade in Canvas than what they have truly earned until the instructor enters a zero score for the missing assignment. Canvas does not treat ungraded missing assignments (in other words, assignments that show a dash in the gradebook cell) as zeroes when calculating student’s total grades. Instead, Canvas ignores all ungraded assignments when calculating a student’s total grade, even those that are past due. When calculating the total grade percentage for the course and each assignment group, Canvas divides the student’s total earned points by a total number of possible points that does not include possible points from ungraded assignments. To make sure students are aware of the impact that missing work will have on their final grade, instructors should regularly enter a score of zero for students who have not turned in an assignment after its due date.

Here is an example of the impact that leaving missing work ungraded in the Canvas gradebook has on total score calculations: imagine a student who has participated in 5 weekly discussions worth 10 points each, earning all 10 points for each discussion (50 points total). Now imagine that a writing project worth another 50 points is past due, and this student has not submitted that assignment. If those five discussions and the writing project are the only assignments in the course to that point, the student will see their total grade as 100% (50/50 points or an A) until the instructor enters a zero for the missing project. When the instructor enters the zero for the writing project, the student’s total grade calculation will update to 50% (50/100 points or an F). The student will not see the impact of the missing project on their total score in Canvas until the instructor enters the zero; if the instructor waits to enter a zero until the end of the term, the student could go through the rest of the course thinking they are in much better standing than they truly are.

Gradebook Zeros Example

While it is easy to do the total grade calculation of this simple example with mental math because it uses a small number of assignments, real courses have greater complexity in grading. Because total grade calculations are often complex, students will struggle to understand and may underestimate the true impact of missing assignments on their grade if those assignments remain ungraded and therefore not included in the calculation of the total grade shown in Canvas. You can help students by entering zeroes right away or as early as it makes sense for your late policy!

New Submission Icon

If your course policies allow students to submit late work, entering a zero score on a missing assignment will not prevent the student from making a late submission. A zero grade is a big attention getter, and seeing the impact the zero has on the total grade in Canvas can motivate a student to make a late submission. Better late than never! To ensure the zero score is not demotivating, make sure students understand that the zero grade you entered is not final. You can use the “Message Students Who” feature in the Canvas gradebook to efficiently send a message to all students with zeros on an assignment which encourages them to submit late and earn (at least) partial credit. Once the student submits the assignment, the Canvas gradebook will show the new submission icon in the cell and update the cell’s status (color) to “Late” (blue). You can grade the late submission and enter a new score to replace the zero.

Entering zeroes for missing work is a crucial step for keeping an accurate and up-to-date gradebook in Canvas, but many instructors learn this step the hard way after receiving a complaint from a student who saw an inflated total grade in Canvas and then got surprised by their official final grade. While entering zeroes is not the only requirement for keeping accurate grades in Canvas, it is a simple-but-not-always-intuitive step that instructors should not ignore. Make sure to do it regularly—ideally while you grade submissions for an assignment. The sooner a student realizes how a missing assignment impacts their grade, the more time they have to compensate. If you would like Canvas to help you keep up with entering zeroes, applying a Missing Submission policy to the gradebook before the start of a course can automate this task for online submission assignments, but note that you may still need to enter some zeroes manually. If reading this article makes you want to have a deeper discussion on setting up and managing your Canvas gradebook, we encourage you to request a CATL Consultation to set up a meeting with a member of our team!

What to Do with Student Evaluation Feedback

The student evaluation of teaching process can produce anxiety in instructors at all career stages. No matter how confident you are in your teaching, being evaluated can feel as though your teaching style, pedagogy, and even personality are being put under a microscope. Turning this anxiety into a productive process for your class can be as simple as reframing how you look at results and what you do with the information provided. This blog post will provide guidance about some best practices for before, during, and after the administration of student evaluations of teaching.

Before and During the Administration of Evaluations

In order to successfully utilize the other suggestions provided in this article, you need to be sure you frame student evaluations effectively with your students. According to Ballantyne, Borthwick, and Packer (2000), students are ambivalent about the usefulness of teaching feedback, but research shows that they believe using feedback for the improvement of teaching is the best possible outcome for completing Student Evaluations of Teaching.  This finding provides a useful place to start in framing your discussion with your students. Several weeks before student evaluations of teaching will be sent out by the university, begin discussing the process with your students. Let them know how they will be evaluating you. Will you provide time in class and leave the room? Will they complete the evaluations on their own time? Will you provide all students with an incentive (e.g., an extra credit point) if the class achieves a certain response rate threshold?

[Note: Check out this other CATL post for more specific ways to boost your student response rates. ]

After you have primed your students to complete the feedback form, inform them how this information will be used. Getting a high rate of thoughtful responses is a goal, and this will likely not happen unless students understand that these responses are important to you and that they will be used to improve your teaching practice. No one wants to complete a task that has no bearing on anything. Students may be led to believe that their feedback on the end of the course evaluation doesn’t hold any weight, so let them know why they should complete it. If you review their comments and make changes to your course, let them know that. If you believe evaluation is important to your career, let them know that. Give your students a reason to want to complete the form. It is important that you are clear that the student evaluation of teaching is important to you. Students are unlikely to find value in an evaluation if you do not.

[Note: Because of the time frame of end-of-semester student evaluations, the feedback they provide can’t be used to improve your current class. This is a reason that doing your own mid-semester evaluation is a great idea, so that you can make course corrections where it seems necessary. See this other blog post for some ideas about how to collect mid-semester feedback.]

Once You Receive Feedback

Don’t focus on the negative

It is human nature to read reviews and focus on one negative comment, even when you have 100 positive comments. You might even find yourself trying to guess at the identity of those outliers who are concerned about one portion of your class or one comment you made, even when the rest of the responses are overwhelmingly positive. This isn’t to say that you should not pay attention to negative comments, but don’t let them define your performance in a class. Don’t overcorrect, especially if you don’t have significant evidence to show that the issue is really a problem. Be aware of the potential for instructor identity-related bias in student ratings, as well (Buser, Batz-Barbarich, & Kearns Hayter, 2022). Give yourself time to think and process through negative comments before deciding what really is the best way to proceed.

Set your standards

Before you look at your feedback, define for yourself what strong performance looks like. For example,  if most of your feedback returns as “somewhat agree” or “strongly agree” on the Likert scale items, you may preemptively decide that would be a positive result. Review the feedback you are given and look for trends. Reflect honestly on your practices if you see areas that come up as consistent areas of weakness. While you cannot please everyone, it is important to note where trends occur that show a potential area for growth.

It’s also important to know that students and faculty often don’t perceive excellence in teaching in the same way. (Spooren et al, 2013). Students may focus on factors that are irrelevant to effective teaching and instead base their evaluations on perceived workload, ability to achieve an “easy A” or any number of factors that don’t actually impact teaching and learning.  For this reason, faculty may ignore feedback as “meaningless,” when it could actually be beneficial to their teaching practice. Be conscious of this potential reasoning trap and think through your feedback, and what you may be able to change.

Don’t try to go it alone

Taking your feedback and meeting with a trusted colleague can be a helpful and beneficial activity. Colleagues, especially those who are more seasoned, will often have experienced many of the issues you are having. If you are disappointed in specific results, talk through your teaching process with colleagues and identify potential areas for growth, or discuss what might have caused the feedback you are unhappy with. Reflecting and talking through these scenarios can help you see clearly what you might want to fix or how to ensure better feedback and a better experience for your students.

Commit to using the feedback

“Student evaluation of teaching (SET) only becomes an effective tool for improving teaching and learning when the relevant stakeholders seriously consider and plan appropriate actions according to student feedback” (Wong & Moni, 2014, p. 397). This passage accurately demonstrates the need for something to happen with feedback once it is collected. It is easy and possibly natural to assume that feedback collected and provided after the end of the semester has limited usefulness. While that feedback cannot provide a better experience for the students who shared it, it can be used to improve your instruction going forward. Consider the input from your students, and be conscious of where that feedback leads you, even if it’s in a different direction for your course. It is not a sign of weakness to accept feedback and make changes. It is the natural progression of an educator devoted to excellence.

Recognize there is an institutional context to evaluation

Although it is natural to focus on course evaluations as a process that occurs between you and your students, the reality is that these tools are used in decision-making, and you are not the only one with access to the results. The standard UWGB course evaluation is used for most classes, and not only you, but administrators such as your Chair and the Associate Provost, have access to the results. Those summaries are used in tenure, promotion, and contract renewal decisions, and UW System also requires gathering student feedback on teaching in our courses (see p. 106 of the Faculty Handbook for policy).

What does all this mean? Well, first, student evaluations are clearly valued at UWGB and beyond. They are not, however, the only means for documenting and improving teaching and learning. The Faculty Handbook also points to other sources, such as peer observation, samples of syllabi and assessments, self-reflection, and engagement in professional development (p. 93). Assemble that information in a teaching portfolio, regularly reflect on your own teaching, invite peer feedback, and participate in educational development opportunities (such as CATL workshops!). Second, since you know others will have access to results, talk to them! Seek out your chair, talk through the evaluations, and learn more about how your unit tends to use them administratively. Finally, remember in the end this should all be about enhancing student learning. If the standard UWGB evaluation does not give you the information you feel you need to improve your course and the student experience, you can always administer a supplemental evaluation on your own. And if you need assistance with any of these issues, remember that CATL is here to help!

Writing Effective Multiple-Choice Questions

Writing good multiple-choice questions is challenging. Tricky or verbose questions can reduce the test item’s reliability and validity, while a poor selection of answer choices can make a question either far too easy or incredibly difficult. A question that suffers several common pitfalls might even work against the learning outcomes it is trying to measure. Fortunately, researchers and assessment experts have identified some common guidelines for creating more equitable and reliable multiple-choice assessments. In this guide, we’ll walk through seven tips for writing more effective multiple-choice test items.

The scope of this guide is focused specifically on authoring multiple-choice questions. If you’d like to dig into when to use multiple-choice assessments, as well as recommendations for scaffolding, testing in online environments, and providing feedback, check out this other CATL blog post on general considerations for creating impactful multiple-choice assessments.

Before getting into the tips on writing questions, we’ll review the anatomy of a multiple-choice question and outline some common language that we use throughout this guide.

Table of Contents

The Anatomy of a Multiple-Choice Item

Throughout this article, we will use the following terms and definitions when referring to the parts of a multiple-choice question:

  • Item: A question and its answer choices as a unit
  • Stem: The posited question that respondents are asked to answer; often phrased as a question, but can also be a statement (e.g., fill in the blank)
  • Alternatives: A list of suggested answers that appear after the question stem; comprised of several incorrect answer options and one (or more) correct or best answer(s)
  • Distractor: An incorrect alternative

An example multiple-choice question in which the top portion is labelled as the “stem” and the answer choices A through F are labelled “alternatives,” with “A” serving as the answer and “B-F” serving as distractors

Example of a multiple-choice item, its stem, and the alternatives

Source: Vanderbilt Center for Teaching and Learning

Tips for Writing Effective Multiple-Choice Questions

Most of the recommendations in this guide have been adapted from How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty Simple (Burton et al, 1991) and Developing and Validating Multiple-choice Test Items (Haladyna, 2004). We’ve distilled down these long-form documents into a few simple guidelines that align with current recommendations from experts at other centers for teaching and learning (see “Additional Resources”). If you are interested in learning more about the research behind these suggestions, we encourage you to check out one or both of the resources linked above.

Tip #1: Tie each item to a learning outcome

In order to maximize an assessment’s validity and reliability, each multiple-choice item should be clearly aligned with one of the assessment’s learning outcomes (and, by extension, the course learning outcomes). Generally, it is recommended to have each item tied to only one outcome each. However, in the case of items that assess higher order thinking and present complex problems or scenarios, it is possible that a multiple-choice item may assess more than one outcome.

Tip #2: Create a specific, clear, and succinct stem

A straightforward, clear, and concise stem free from extraneous information increases a multiple-choice item’s reliability. When writing and revising your question stems, it is a good practice to ask yourself if there is a simpler or more direct way to rephrase a question. Overly wordy stems rely on students’ reading comprehension, which is usually not one of the intended outcomes of the assessment. Likewise, confusing or ambiguous stems can be accidentally misleading. Ideally, a student who has mastered the target outcome should be able to answer the question posited even without the alternatives present.

For millennia, humanity has been entranced by the ebb and flow of the tides. Many past civilizations believed the ocean's waters were controlled by monsters, spirits, or gods, but today know the scientific laws and theories that explain the tides. These movements are influenced by, in part, the gravitational force from the sun, the earth’s rotation, shoreline geography, and weather patterns, but all of these pale in comparison to the effects of:

  • a)  El Niño
  • b)  The gravitational force of the moon
  • c)  The ozone layer
  • d)  Deep-sea trenches

(Answer: B)

Why it doesn’t work: The extra information in the question stem makes it difficult for the test-taker to discern the question that is being posed. The question itself is also worded ambiguously.

Earth’s tides are influenced primarily by:

  • a)  El Niño
  • b)  The gravitational force of the moon
  • c)  The ozone layer
  • d)  Deep-sea trenches

(Answer: B)

Why it works: The question stem has been revised to remove all unnecessary information and it now poses a simple, straightforward question.

Tip #3: Avoid using negatives in question phrasing

It is usually best to avoid negative phrasing in question stems, such as asking students to identify which alternative does not belong. Negatively phrased stems tend to be less reliable in assessing students’ learning than stems that ask students to identify the correct answer. The exception to this guideline is in cases when knowing what not to do is key, such as questions related to safety protocols. If you do choose to include a negative qualifier, use bold or italics to emphasize the negative word and make sure that you don’t create a double negative with any of the alternatives.

Which of the following is not a quality of an active listener? 

  • a)  Not talking over others 
  • b)  Making eye contact with the speaker 
  • c)  Asking clarifying questions 
  • d)  Mentally planning a rebuttal while the other person is speaking 

(Answer: D)

Why it doesn’t work: The question stem is phrased in the negative and the negative qualifier is not emphasized, making the question less reliable. Additionally, one of the alternatives also contains the word “not,” creating a double negative with the question stem. 

True or false? An active listener…

  • Refrains from talking over others (T/F)
  • Makes eye contact with the speaker (T/F)
  • Asks clarifying questions (T/F)
  • Mentally plans a rebuttal while the other person is speaking (T/F)

(Answers: T, T, T, F)

Why it works: This question stem has been rephrased to avoid using the word “not.” The answer choices have been turned into four separate true/false statements so each item can be assessed separately and have been revised to remove the word “not.”

Which of the following is not a recommended action to protect yourself during an earthquake if you are inside a building?

  • a) Drop to your hands and knees
  • b) Take shelter under a sturdy nearby desk or table
  • c) Crawl to the nearest exit
  • d) Cover your head and neck with your arms

(Answer: C)

Why it works: In this scenario, knowing what not to do during an earthquake is one of the learning outcomes, so it is appropriate to use a negative qualifier. The negative qualifier in the stem, “not,” has also been emphasized with bold and italics to draw attention to it.

Tip #4: Use plausible distractors

Good distractors need to appear plausible to students that have not met the target learning outcome, but not so tricky that they could be argued as correct answers by a test-taker that has met the learning outcome. When you are writing a multiple-choice question it is often useful to write the stem first, then the correct answer first. Once you have decided on these two pieces, formulate 2-4 distractors based on common student misconceptions. If you can’t think of another “good” distractor for a set of alternatives, it is usually better to have fewer alternatives than to include extra alternatives just for the sake of consistency.

George Washington Carver is best known for his work as a(n) ______.

  • a)  Agricultural scientist
  • b)  Extraterrestrial expert
  • c)  Basket-weaver
  • d)  Juggler

(Answer: A)

Why it doesn’t work: The distractors are so absurd and far-removed from the topic of the question that even a student who knows nothing about George Washington Carver could discern the correct answer, making the test item neither reliable nor valid.

George Washington Carver is best known for his work as a(n) ______.

  • a)  Agricultural scientist
  • b)  Electrical engineer
  • c)  Microbiologist
  • d)  Politician

(Answer: A)

Why it works: The distractors seem plausible, creating a question that will more accurately assess students’ knowledge of George Washington Carver.

Tip #5: Use homogeneous phrasing and formatting for alternatives

Small typos, inconsistencies in tenses or phrasing, or changes in text formatting can accidentally provide clues about which alternatives are the distractors and which are correct answers. Savvy test-takers can pick up on these inconsistencies and use this information to deduce the correct answer even if they have not achieved mastery for the desired outcome, so keep an eye out for these things as you proofread your exam. If you notice formatting inconsistencies in your Canvas quizzes, you can use the Rich Content Editor to remove all formatting and set the selected text to Canvas’s defaults.

What three parts of speech can an adverb modify?

  • a)  Verbs, adjectives, and other adverbs
  • b)  Noun, adjective and preposition
  • c)  Verb, noun and conjunction
  • d)  Adjective, adverb and exclamation

(Answer: A)

Why it doesn’t work: Answer choice “A,” the correct answer, is in a different font from the other alternatives. Additionally, the distractors use the singular version of each part of speech, rather than the plural, and omit the Oxford comma before “and.” These inconsistencies hint to students that “A” is the odd one out.

What three parts of speech can an adverb modify?

  • a)  Verbs, adjectives, and other adverbs
  • b)  Nouns, adjectives, and prepositions
  • c)  Verbs, nouns, and conjunctions
  • d)  Adjectives, adverbs, and exclamations

(Answer: A)

Why it works: The distractors have been revised to look consistent with the correct answer, creating a question that assesses students’ knowledge of parts of speech, rather than their eye for detail.

Tip #6: Avoid using none-of-the-above or all-of-the-above as alternatives

Questions that provide “all of the above” or “none of the above” as alternatives are generally less reliable for assessing outcomes than a multiple-choice question with mutually exclusive alternatives. The table below outlines the use cases for “all of the above” and “none of the above” along with why they are flawed for reliable assessment in each instance. If you can’t think of another distractor while drafting a question, remember that it is okay for some questions to have fewer alternatives.

Use of “all of the above” and “none of the above” 

Alternative

Weakness
“All of the above” as the answer Can be identified by noting that two of the other alternatives are correct
“All of the above” as a distractor Can be eliminated by noting that one of the other alternatives is incorrect
“None of the above” as the answer Measures the ability to recognize incorrect answers rather than correct answers
“None of the above” as a distractor Does not appear plausible to some students

(Adapted from How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty, Brigham Young University)

Tip #7: Create questions with only one correct alternative

Like none- or all-of-the-above alternatives, asking students to identify multiple correct alternatives is a less reliable form of assessment than an item with only one correct answer. Multiple-response questions are also reliant on confusing grading calculations, since selecting an incorrect alternative “cancels out” a correct selection (this Canvas guide goes into more detail about how Multiple Answer questions are auto-graded). And, in questions with more incorrect than correct answers, students can still score points by selecting no answers at all!

A straightforward multiple-choice item with only one correct answer and mutually exclusive alternatives is a more reliable way of discerning whether a student truly knows a concept or is guessing. Another option is to turn a multiple-answer question into a series of true/false questions, which will provide a more reliable picture of students’ understanding and a more valid grade for their efforts.

Check all that apply. COVID-19:

  • a)  Is an infectious disease
  • b)  Is spread primarily through fungal spores
  • c)  Can be treated with antibiotics
  • d)  Can infect people of all ages

(Answer: A and D)

Why it doesn’t work: Because of the way multiple-response questions are graded, they are less reliable than individual multiple-choice or true/false questions.

True or false? COVID-19:

  • Is an infectious disease (T/F)
  • Is spread primarily through fungal spores (T/F)
  • Can be treated with antibiotics (T/F)
  • Can infect people of all ages (T/F)

(Answers: T, F, F, T)

Why it works: Each statement is assessed individually, allowing for more granular and accurate scoring.

Questions?

Want more tips for writing multiple-choice questions? Looking for someone to help brainstorm outcome-aligned questions with? CATL is here for you! Reach out any time to set up a meeting or send us your questions at CATL@uwgb.edu.

Additional Resources