Its purpose is to support student learning, but Canvas, Indiana University's online learning-management system, is helping academic advisors learn something of value too. Using data from Canvas to identify students who may need academic support, IUPUI Campus Career and Advising Services has designed an early-intervention program that's improving student success.
The intervention, launched last fall as a yearlong pilot study in University College, is based on activity and engagement in Canvas -- behaviors that often correlate to student performance.
Capturing student-specific data across all of their courses, including number of assignments due, number of submitted assignments and overall minutes spent in Canvas, the advising team calculates a "Canvas engagement score" for each student. Individuals with scores falling within the lowest quartile for engagement in their school or unit are contacted by their academic advisor for a check-in.
Scores are calculated in two-week intervals during the first six to eight weeks of the semester, offering three opportunities for students matching the criteria to be contacted.
"Students receiving the advising intervention last fall had a term GPA that was on average 20 percent higher than our control group, and their persistence rate at the institution was higher as well," said Matt Rust, senior executive director of Campus Career and Advising Services. "Our goal was to determine whether this intervention is effective, and we've certainly seen enough success in this first semester to scale up the effort, not only in University College, but also in sharing the results with other IUPUI schools and IU campuses."
While the pilot study shows there is significant predictive value tied to Canvas engagement, Rust said looking exclusively at engagement in the online system inherently results in a few "false positives" -- flagging students, for example, who are on track academically but not regular users of Canvas. Because the data capture doesn't include grades and doesn't account for optional assignments, the formula for identifying the "right" students for the intervention isn't foolproof. Even with these limitations, the false-positive rate was below 5 percent for the fall semester.
"We expect to make some modifications for outreach based on what we learn, but we don't need it to be perfectly predictive," Rust said. "We just want to get an idea, based on some reliable engagement criteria, of who we should be checking in with during the opening weeks of the semester."
In fact, Rust said, even in cases when students who don't need the Canvas intervention have been contacted, advisors have been able to make valuable connections that could lead to future interactions.
"Reshaping the understanding of advising is definitely a big part of this," he said. "With a more proactive approach, advising becomes a comprehensive set of conversations that focus not just on academic planning and policies, but career planning, engagement in high-impact practices, use of campus resources, and intellectual and personal development."
Part of IUPUI's proactive advising strategy
The Canvas engagement intervention is the most recent initiative in a larger proactive advising strategy launched at IUPUI in 2018.
For the past three years, the campus has conducted friendly interschool competitions aimed at increasing advisor meetings with undergraduates -- particularly freshmen and sophomores -- during the first seven weeks of the fall semester, a crucial period for addressing potential barriers to academic success.
In 2019, University College revamped its advising model, focusing on coaching-style conversations and an expectation that all first-year students meet multiple times with their advisor. This spring, the School of Engineering and Technology launched a similar initiative, emphasizing coaching and more frequent outreach, within its new student advising center.
"Regular interaction, a primary aspect of proactive advising, enables our advisors to better support student needs and address their challenges in real time," said Jay Gladden, associate vice chancellor for undergraduate education and dean of University College. "In contrast to more-traditional advising models that focus solely on course selection, proactive advising covers multiple aspects of the student's experience, helping advisors obtain a better understanding of needs and connect students to resources."
Since IUPUI's shift to proactive advising, University College, the academic unit with the most first-year students, has been successful in meeting with many of its students during the fall.
In the fall semesters of 2018 and 2019, advisors were able to meet with about 90 percent of University College's more than 4,500 students. Despite the pandemic and the resulting hiring freeze that has left some advisor positions vacant, University College was still able to meet with more than 76 percent of its students in fall 2020.
"We know there are many students who won't seek help for a variety of reasons and therefore are at risk of slipping through the cracks," said Cathy Buyarski, executive associate dean of University College. "Measures like the Canvas engagement intervention, and proactive advising in general, ensure students are connected with the advising assistance they need."
IUPUI recently received a grant from the Association of Public and Land-grant Universities to lead, along with the University of Louisville, a study related to the implementation of proactive advising at metropolitan universities in the United States.