Making Impactful Use of Canvas Analytics in Your Course

Like many websites, Canvas collects data from users as they navigate their courses. Thankfully, unlike many websites, Canvas collects this data not for the purpose of selling it to advertisers but for the purpose of presenting it to instructors. Canvas presents collected student activity data in a course page titled “New Analytics,” which contains charts and tables designed to help instructors make use of this data. While New Analytics contains well-organized representations of course data, it does not prescribe specific actions or provide a formula for making use of the data. In this post, we summarize the data available in New Analytics and recommend ways you can interpret it to take actions in your course that can help improve student outcomes. 

Detecting Course Trends 

New Analytics Window

New Analytics can help reveal trends in student achievement from assignment to assignment and student engagement from week to week. After launching New Analytics from the course navigation menu or the button on the right side of the course Home page, you’ll see a series of tabs across the top of the page. The first two tabs, “Course Grade” and “Weekly Online Activity” have data views that can help you identify course trends. The Course Grade tab has a chart which shows the average grade for each assignment in your course. Each assignment in your course will be represented by a dot on this chart. The dot’s position on the y-axis represents the average grade for that assignment. A quick glance at this chart can help you identify the assignments where the class atypically excelled or struggled and help you confirm—or refute—suspicions you developed about performance trends while grading assignments. Thinking critically about why the class might have been more or less successful on a particular assignment can lead to ideas for course design improvements. A close look at a successful assignment may lead to insights on what works well in a course; a close look at a less successful assignment may reveal a need to incorporate scaffolding assignments and additional support. Clicking on an assignment’s dot on the chart will reveal additional statistics, including a grade distribution chart and the number of missing and late submissions. 

Clicking the Weekly Online Activity tab will show a chart of the average page views and course participation actions during each week of the course. Viewing this chart can help you identify whether engagement with your Canvas course is waning, holding steady, or growing. Beneath the chart is a table of course resources which shows how many students have viewed each item, how many overall views it’s received, and how many times a student has participated (a list of the actions Canvas counts as a “participation” can be found in this Canvas guide). You can sort this table by any of its columns to identify which elements of your course get the most and least engagement. If an important resource in your course isn’t garnering as many views as you’d like it to, ask yourself “why?” and consider ways to either guide your students to that resource or phase it out and incorporate its key content into the resources your students are reliably viewing (Clum, 2021). Look at the resources that have gotten the most views and participation and check for commonalities to gain insight on what captures your students’ attention. You can click on any data point in the Weekly Online Activity chart to open a panel that shows activity data filtered for that specific week. The data in this panel can give you an idea of whether students are keeping up with the pace of your course or whether they are still working through older resources. 

Checking on Individual Students 

New Analytics can also help you identify students who may benefit from an intervention from a professional adviser because they have disengaged with your course or never engaged at all. The Students tab of the New Analytics page shows a table with the following statistics for each student: 

  • Current grade 
  • Percentage of assignment submissions made on time 
  • Last date of a participation action 
  • Last date the student viewed any page in your course 
  • Count of total page views 
  • Count of total participation actions 

You can click any column header on this table to sort the table by that column. Looking at this table during the first few weeks of a term and sorting it by “Page Views” can help you quickly identify students who have not engaged with the Canvas course. Students with no or very low page view counts have not engaged with your course. You can issue an ad-hoc alert in EAB Navigate to request that UW-Green Bay’s professional advising team reach out and help set a student on a path to academic success. 

Clicking on a student’s name in this table will open a student-specific data view that shows that particular student’s assignment grades and weekly activity over time. If you’ve noticed a downturn in a student’s performance or engagement, this view can help back up your observations with data. Comparing a student’s assignment grades or activity with the class average can help you contextualize any trends you see. You can view an individual student’s data alongside the class average on the same chart by adding that student to the filter field above the chart on the Course Grade or Weekly Online Activity tabs.

Sending Smart Messages 

Sending Smart Messages in Canvas

New Analytics also makes it easy to send messages to students who fit certain performance or activity criteria. As you explore the Analytics tool in your course, keep an eye out for the message icon that can be found on most of the tabs and panels. Clicking this icon will begin composing a Canvas Inbox message which you can send to students that meet a customizable criterion related to an assignment grade, weekly activity, or engagement with a specific resource. Here are a few examples of the types of messages you can target through New Analytics: 

  • Check-in with students who haven’t yet viewed the course this week 
  • Send congratulations to the students who did well on an assignment 
  • Encourage a growth mindset for students who struggled with an assignment and point them to helpful resources 
  • Remind students who have missing assignments to make a submission 

These quick instructor encouragements and interventions can help your students stay engaged with the course and on-target to reach their goals (Bostwick & Becker-Blease, 2018). Especially in online asynchronous courses, sending these targeted check-in messages can help establish your presence and ensure that students know you care about their success. 

Try It Out!

Coupling the data in Canvas New Analytics with the observations you make while teaching can help you make accurate judgments about what works well and not so well in your course. It can also help you identify when a student needs some additional support, and the incorporated messaging tool makes it easy to follow-up. We encourage you to open the New Analytics page in your Canvas courses, explore the data within, and ask yourself whether what you see aligns with your assumptions of how students experience your course. Try sending a congratulatory message to the students that excelled on an assignment and a friendly reminder message to the students who owe you work. We’d love to hear about your experience exploring and interpreting the data! Please feel free to reach out to us at CATL@uwgb.edu to tell us your story, ask a question, or request a consultation!

References 

  • Bostwick, K. C. P., & Becker-Blease, K. A. (2018). Quick, Easy Mindset Intervention Can Boost Academic Achievement in Large Introductory Psychology Classes. Psychology Learning & Teaching, 17(2), 177–193. https://doi.org/10.1177/1475725718766426 
  • Clum, K. (2021, May 14). Using canvas analytics to support student success. KatieClum.org. Retrieved December 5, 2022, from https://katieclum.org/2021/05/14/using-canvas-analytics-to-support-student-success/ 

Evidence-Based Frameworks and Strategies for Keeping Students Engaged

Keeping students engaged in their learning throughout an entire semester is a challenge that exists across all disciplines and modalities. Though the ways in which you implement strategies for increasing student engagement might vary because of these factors, the good news is that the underlying principles remain the same. Below are some of the key methods and strategies that have emerged as common themes across many studies on the relationships between teaching practices and student engagement.

Foster a Culture of Growth, Trust, and Belonging

Part of a student’s engagement in a course is tied to the affective domain of learning, or a student’s thoughts and feelings about their own learning. Does the student feel like they belong in this learning environment? Are they respected by their peers and the instructor? Do they see their instructor as an ally in the learning journey, or as an adversary?

One aspect of the affective domain is whether an individual has a growth mindset or a fixed mindset. The Center for Learning Experimentation, Application and Research at the University of North Texas has a great list of growth mindset interventions instructors can implement. It is worth noting, however, that research seems to indicate the effectiveness of these interventions is contingent on the instructor’s mindset as well. Studies have shown that instructors with a greater growth mindset (as opposed to a fixed mindset) have smaller racial achievement gaps and inspire more student motivation in their courses.

The affective domain also includes students’ feelings of belonging and trust. While the degree to which you can affect these feelings has limitations, evidence-based practices usually boil down to how you interact with students and facilitate interactions between students. A few examples include using a welcoming tone in your syllabus, modelling inclusive language, and taking the time to get to know your students’ names. Even in asynchronous classes it is important to build trust with your students. For example, you might want to consider using a week-one survey to provide your students with an opportunity to tell you about themselves.

Break Up Lectures & Add Opportunities for Active Learning

When there is a lot of content that needs to be disseminated across the duration of the semester, lectures are a common method for communicating that information quickly and efficiently. But the longer and denser the lecture is, the more instructors risk losing their students along the way due to cognitive load.

One solution is to build pause points into your lectures. Students benefit from structured pauses during lectures as it allows them space to question, process, and reflect on the information that they’ve absorbed. For pre-recorded lectures, the same idea can be achieved by breaking up a long lecture video into multiple short, topical videos (research suggests 6-12 minutes is an ideal length for maintaining student engagement). Fortunately, Kaltura (My Media) makes it very easy to trim and save video clips from right within Canvas.

When adding pauses for students to digest information, it is also beneficial to create opportunities for active learning activities. These activities can be very brief, such as using an anonymous polling tool to check for student understanding during a lecture. For more in-depth active learning, consider making time for small group discussions, written reflections, and other exercises that require students to employ higher order thinking skills. For courses with an asynchronous component, PlayPosit allows instructors to add a variety of engagement activities to pre-recorded lecture videos, while Hypothesis may be useful for incorporating annotation and reflection activities into assigned readings.

Provide Transparency and Support

When a student needs to spend a lot of mental energy figuring out the logistics of how to complete an activity, they have less mental energy left to engage with the course materials themselves. Therefore, transparency and scaffolding are both key elements to designing engaging assignments.

The Transparency in Learning and Teaching (TiLT) framework is designed to help instructors write clear and descriptive instructions for learning materials and assignments. For this framework, lay out the task, purpose, and criteria for each learning activity. If a student knows what they are supposed to do, why they are supposed to do it (how it ties to the course learning outcomes), and how they are going to be assessed, they can go into the activity more confident in their ability to engage with it.

It is common for a student to stop engaging with a course if they feel like they don’t have the means or resources to complete the tasks they’ve been assigned. Proper instructional scaffolding can help counter this issue by bridging some of the cognitive gaps and reducing the number of students that fall through the cracks. For example, if the final assignment in your course is an 8-page research paper, consider breaking up the process into several smaller assignments, such as having students submit their topic, bibliography, and outline at various points throughout the semester. Other ways to provide scaffolding this assignment might include modelling (providing examples of papers that meet the outcomes of the assignment), incorporating instructor or peer feedback for the outline or an early draft of the paper, and providing a robust rubric to guide students on how to meet the assignment outcomes.

Additional Resources

Engaging students is a broad topic that we are only just able to scratch the surface of in this post. Below are some resources for further reading if you’d like to dive in deeper.

Questions?

As always, we welcome you to share your ideas for engaging students by dropping a comment below or emailing us at CATL@uwgb.edu. If you’d like to discuss any of these methods or ideas one-on-one, a CATL member would be happy to meet with you for a consultation as well.

Evidence-Based Approaches to Promoting Academic Honesty

Academic honesty has always been a concern in higher education, but the proliferation of technology has changed the scope and nature of the problem. Students have access to more electronic means to cheat, including AI-generated papers and websites that provide access to test bank questions and answers. Meanwhile, professors can deploy competing technologies designed to search automatically for plagiarized content, lock down browsers during exams, or remotely proctor test-taking.

It should come as no surprise that there are ethical concerns about both academic dishonesty itself and the privacy and intellectual property issues raised by technologies intended to detect or prevent it. In fact, one Canadian professor recently taught an academic course on cheating, and he is a co-investigator on a large-scale study of college student motivations to pay others to do their work.

The SoTL literature on this topic often lags behind the technological advances, but there are some recent studies instructors may find helpful. Duncan and Joyner (2022) surveyed students and TAs about digital proctoring, and although their sample was not representative, their resulting article is definitely worth a read. They provide a nice overview of costs of benefits of the practice, and they also effectively summarize the literature on alternative assessment strategies faculty can employ. Another recent addition to the body of knowledge on academic honesty is a study of six relatively low-tech and brief methods to reduce cheating, such as allowing students to withdraw assignments. Again, there are some methodological issues with the research, but instructors may find the techniques and review of past research on them illuminating.

The issue of academic integrity is complex, multi-faceted, and rapidly evolving given its intersection with emerging technology. Additional examples of relevant SoTL research on the topic are included below. CATL will update this list as we are able. Feel free to contact us with suggested resources as well.

Additional Resources

Resources for Incorporating Student Feedback

Blue note card with the word Feedback

Though further study is needed, research into the efficacy of regularly collecting and incorporating student feedback has indicated that it is a valuable practice for improving as educators (Mandouit, 2018). Instructors can use a variety of methods for gathering student feedback to shape and tune their instruction, from informal mid-semester surveys to a final course evaluation. This blog post is a collection of past resources that continue to hold up as helpful resources for instructors looking to integrate meaningful student feedback into their courses.  

Mid-Semester Feedback 

Collecting mid-semester feedback provides instructors with insights into how students are engaging in their courses and is a great opportunity to discover any areas for instructional improvement before end-of-semester evaluations. To help instructors incorporate mid-semester feedback in their courses, CATL created a blog post on collecting mid-semester feedback from students which includes student feedback survey examples that instructors can download and use in their Canvas course. Survey types include short surveys gauging students’ class preparations and experiences, start-stop-continue surveys, open-ended questions asking students what has helped or hindered their learning, and the Plus/Delta Survey that asks students to provide what they like in the course and what they would change.  

To see a demonstration on how to build a mid-semester survey in Canvas, view this short CATL video resource, TeAch Tuesday – Mid-Semester Evaluations. This video also dives into some best practices for mid-semester evaluations. For example, you might consider sharing with your students any trends you spotted in the feedback. This can help provide justification for certain aspects of your course that you plan on changing or keeping the same (e.g., “the majority of you asked for more review time, so I am going to build more time into our schedule before exams to review as a class.”). For an even deeper exploration of mid-semester evaluations, you can watch this PlayPosit bulb from a workshop CATL ran in 2021 on collecting and working with mid-semester feedback. Topics covered in the workshop include why instructors collect mid-semester feedback, approaches for enhancing the quality of feedback questions and responses, and a breakdown of different surveys and evaluations instructors can use.  

After you have collected your mid-semester feedback, you need to decide what to do and how to move forward with your students’ responses, which can often be the most difficult part. In addition to the CATL workshop linked above, the guide, What to Do with Mid-Semester Feedback, created by the Center for Teaching at Vanderbilt University, is a great resource for thoughtful reflection on and implementation of student feedback. Their guide includes recommendations on how to include students in a discussion of the survey results, reflect on student comments, and use survey data to identify patterns in your instruction and students’ learning.  

End-of-Semester Course Evaluations 

Compared to mid-semester evaluations, end-of-semester student evaluations of instruction (commonly referred to as course evaluations) are a more formal method of collecting student feedback that also relies on standardized forms and questions. Still, it can be a useful tool for instructors to reflect on their teaching when given careful thought and consideration. Since Fall 2021, most programs have adopted a new, standardized student course evaluation form that was approved by UWGB Faculty Senate. Though the form is used for most instructional courses, your department may use its own form or evaluation process, so check with your department chair if you have questions. To help instructors explore and understand this new standardized form, CATL created a video overview of the new course evaluation form. 

Once students have submitted their final course evaluations, what can instructors do with the student feedback on their instruction? The video linked above on the new course eval form includes some practical advice on interpreting your student feedback. Some tips include looking for patterns and trends in the data, taking the feedback seriously, but not personally, and ignoring negative outliers. In general, most of the concepts in the Vanderbilt guide and CATL workshop recording on mid-semester feedback also apply, though in the case of course evals you will have to wait until the next iteration of your course and cohort of students to implement the feedback. 

We’d love to hear from you! 

Gathering and using feedback, whether it’s informally over the semester, at the mid-semester mark, or with an end-of-semester evaluation, can be a useful tool for developing your teaching and courses in a variety of ways. Let us know how you collect feedback and work it into your courses, and share your success, advice, and methods you’ve found most effective. Feel free to drop a public comment or email us at catl@uwgb.edu.  

If you would like to discuss more about how to collect and evaluate course feedback or how to use other methods of feedback and reflection in your instruction, please reach out to CATL to schedule a consultation!  

“Required” Reading: Guiding Students Through Your Course with Canvas Module Requirements

When provided with a task, almost invariably, some students will overlook or ignore provided instructions, skip past foundational lessons, and end up taking their own “creative” approach to completing it. While some students can find success going their own way, when a student misses instructions, they often end up making life harder for themselves and their instructor. When teaching with Canvas, you have access to tools that can help add order to how students progress through your lessons, instructions, and assignments. When you organize and present your content in modules, you gain the ability to add requirements that require each student to view and/or interact with specified course items before they can progress and access items positioned further down in the course module order. You can ensure that your students’ route to an assignment goes through the important scaffolding pages. While they are not a panacea and should be applied with careful thought so that they act as a guide and not an unnecessary obstacle for students, implementing module requirements can help prevent students from starting a task before they have engaged with preparatory lessons and activities.

Forcing top-to-bottom progression within a module

Within a single module you can use requirements to force students to progress through that module in top-to-bottom order. These requirements can help enforce that students, for example, view a page with important instructions before they can access and submit to an assignment. To set up a module with requirements that students must complete in order, click the module header’s Options icon, then click Edit to open the Edit Module Settings menu. In this menu, select + Add Requirement to begin adding your first requirement to the module and reveal additional options. Enabling the Students must move through requirements in sequential order checkbox will lock each module item until all the requirements above that item in that module have been completed. This forces students to work through the module requirements in top-to-bottom order. You then need to add requirements to the module. You will need to add one requirement for each module item students need to complete in order.

You add a requirement by selecting + Add Requirement, and then you configure a requirement by making selections in two drop-down menus. In the left drop-down menu, select the module item to which you want to add the requirement. In the right drop-down menu, select how the students must complete that requirement. Depending on the type of the module item, this second drop-down menu will show several options (click each option below to expand it and reveal suggested uses):

Only requires students to open and view the module item. This is a simple requirement well-suited for course Pages.

Requires students to open the item and then click a Mark as done button at the bottom of the page. If you use this type of requirement, make sure to provide students with instructions for marking items as done. Students can overlook the button and feel stuck if they’ve never encountered this requirement before.

Only used for Discussions and Pages that are set to allow students to make edits. Students will satisfy this requirement once they have posted a reply in the Discussion or saved an edit to the Page. Avoid picking this requirement for pages which can only be edited by Teachers, as students will not be able to complete the requirement.

Requires the student to make an Assignment submission, make a reply in a graded Discussion, or complete a Quiz attempt.

Requires students to earn at least the minimum score on the graded item, which you designate when setting up the requirement. This type of requirement works well with Quizzes that allow multiple or unlimited attempts.

Adding a requirement for each module item that contains important information or a required task will ensure students engage with the content in your intended order.

Locking a module until the previous module has been completed

Requirements can also be leveraged to lock an entire module until the student completes the requirements of one or more modules above it. Controlling module-to-module progression is done with the additional step of adding module prerequisites. You can add one or more prerequisites to a module through the same Edit Module Settings menu where you add requirements. When adding a module prerequisite, you select an entire module that appears above the module you are currently editing. Adding one or more prerequisites to a module will prevent a student from accessing all content in that module until they have completed the requirements of each module that is set as a prerequisite. Any module that is selected as a prerequisite must contain at least one requirement—Canvas needs to know the criteria for completing that module and satisfying the prerequisite. A module that is selected as a prerequisite but has no requirements will have no effect on course progression.

Screenshot of the Edit Module Settings menu: “Module 1” is selected as a prerequisite for “Module 2”
Edit Module Settings menu: “Module 1” is selected as a prerequisite for “Module 2”

Here is an example of how you can set this up in a course. Let’s say that the content in Module 1 is so fundamental to the content in Module 2 that you want to prevent students from viewing anything in Module 2 until they’ve fully engaged with the content in Module 1. Imagine that both Module 1 and Module 2 contain a Page, a Discussion, and an Assignment. Your first step in forcing students to complete Module 1 before accessing Module 2 is to edit Module 1 and add three requirements:

  1. Module 1 Page – view the item
  2. Module 1 Discussion – contribute to the page
  3. Module 1 Assignment – submit the assignment
Screenshot of the the Edit Module Settings menu with the example requirements for Module 1 set up
Edit Module Settings menu with the example requirements for Module 1 set up

Adding these requirements tells Canvas how to determine whether a student has completed Module 1: a student has completed Module 1 once they have viewed the Page, made a reply in the Discussion, and submitted to the Assignment. The next step is to edit Module 2 and add a prerequisite, selecting Module 1 as the prerequisite module. That is all that is needed to force students to complete Module 1 before accessing Module 2; it is not necessary to add any requirements to Module 2 unless you plan to use Module 2 as a prerequisite in a subsequent module. You may still want to add requirements to Module 2 just for the additional visual guidance and feedback they provide to students.

Screenshot of two example modules with requirements and prerequisites set up
Two example modules with requirements and prerequisites set up

Use Cases

There are many creative ways you can leverage module requirements and prerequisites to exercise control over the flow of your course. Here are a few illustrative use cases (click each to expand it):

You can require students to acknowledge the policies and essential information contained in your syllabus before they can access the rest of your course by creating a syllabus quiz and setting up module requirements and prerequisites. First, add a quiz to the “Introduction” module at or near the top of your Canvas course and add any number of objective questions to test your student’s knowledge of important syllabus content. Set up the quiz to allow for unlimited attempts and to keep the highest score. Next, edit your “Introduction” module to add a requirement that students score at least X points on the Syllabus Quiz. “X” can be whichever minimum score you deem good enough to allow progress through the course. Finally, edit each subsequent module of the course to add the “Introduction” module as a prerequisite. This setup requires students to take (and retake) the Syllabus Quiz until they score at least X points before they can access any of the content after the “Introduction” module.

You can set up requirements within a single module to have students complete a pretest quiz before going through the module content and then take a post-test quiz after completing the module content. This process could help you evaluate the effectiveness of your instruction and/or apply a metacognitive approach to help students gain awareness of their learning and gaps in their knowledge. To create this setup, add a pretest quiz at or near the top of a module and a post-test quiz at or near the bottom of the module and put lesson content and additional formative assessment activities in between the two quizzes. Edit the module to add a requirement to each module item (including requirements to “submit” to each of the two quizzes) and enable the Students must move through requirements in sequential order checkbox. Students will need to progress through the module in top-to-bottom order, first taking the pretest at the top of the module, then engaging with the content, and finally taking the post-test at the bottom of the module. You can set your pretest quiz to be a “Practice Quiz” so that scores are not added to the gradebook.

Even if you don’t need to force a linear progression through your course modules, adding requirements to your modules automatically adds visual feedback that helps to communicate expectations to students and helps students track their own progress through the course. Any module item that is used in a module requirement will display its requirement underneath its title within the module. This is a simple automatic piece of visual feedback that can help students keep track of their tasks. Students also see additional indicators of their progress. After a student completes a requirement, the module item is marked with a green checkmark to signal completion. If a student has started a module but has not yet completed all its requirements, that module’s header is marked with a red circle; once the student completes all of a module’s requirements, this indicator changes to a green checkmark. Instructors can monitor student progress through module requirements by clicking the View Progress button located above the first module of the course.

The visual feedback provided by module requirements adds a light element of gamification to your course, turning each module into a list of sub-missions to be completed. Requirements can be further leveraged to add game-based learning elements to your courses. Use requirements to set minimum scores on low-stakes quizzes that allow multiple attempts and unlock harder “levels” of your course once a student achieves the target score. You can further add the Canvas Badges (Badgr) integration to your course to award “achievements” for the completion of modules and (optionally) enable an anonymous leaderboard to foster competitive motivation among students.

Conclusion (Prerequisite: Read All Above Sections)

We hope this blog post has given you a sense of how and when you can use module requirements in your Canvas courses. When used thoughtfully, module requirements are an effective tool for encouraging students to move through your course in the sequence you intended. Recall from your past teaching that assignment or a module in a course where students got off-track because they somehow managed to miss vital information that was right there for them. Next time you teach that course, try employing requirements to funnel students through that important supporting content before they can start work on the assignment. If you have an idea for employing module requirements in your Canvas course and would like to discuss how to best put it together, please reach out to catl@uwgb.edu or request a CATL consultation to meet with a member of the CATL team!