Big Data

Every time University of Arizona students swipe their ID cards — at the student union, the rec center, the library — they leave a digital trace, showing exactly where they’ve been and when.

One UA researcher is tracking those digital traces to see what they reveal about students’ routines and relationships — and their likelihood of returning to campus after their freshman year.

Sudha Ram, a professor of management information systems, directs the UA’s INSITE: Center for Business Intelligence and Analytics in the Eller College of Management. The center focuses on harnessing the power of big data, using machine learning and network science, to help businesses and organizations make better-informed decisions.

The goal of Ram’s Smart Campus research is to help educational institutions re-purpose the data already being captured from student ID cards to identify those most at risk for not returning after their first year of college.

“By getting their digital traces, you can explore their patterns of movement, behavior and interactions, and that tells you a great deal about them,” Ram says.

Freshman retention is an ongoing challenge for public universities nationwide. It’s important not only for the obvious reason — that a university’s goal is to educate students — but also because retention and graduation rates influence a university’s reputation and national rankings. And students’ first two years in college have been found to be critical to their likelihood of completing a degree.

Traditionally, factors such as academic performance and demographic information have been heavily relied on to predict which students are most at risk for dropping out. Ram’s research takes a different approach, focusing on students’ interactions and campus routines.

Using ID Cards to Track Social Interactions

Students at the UA are issued CatCard student IDs when they enroll. They use the cards at locations including residence halls, the Student Recreation Center, various campus labs, the library and the Think Tank academic support center, to name a few.

Many students also load cash onto the card for use in vending machines and to pay for food and services at the Student Union Memorial Center. The total number of campus locations that accept CatCards is near 700.

“It’s kind of like a sensor that can be used for tracking them,” Ram says of the card. “It’s really not designed to track their social interactions, but you can, because you have a timestamp and location information.”

For example, if Student A on multiple occasions uses her CatCard at the same location and at roughly the same time that Student B uses her card, the pattern suggests a social interaction between the two.

Working in partnership with UA Information Technology, Ram gathered and analyzed data on freshman CatCard usage over a three-year period. She then used that data to create large networks mapping which students interacted with one another and how often. She also looked at how students’ interactions changed over time by constructing networks representing two weeks of data at a time over a 12-week period.

“There are several quantitative measures you can extract from these networks, like the size of [a student’s] social circle, and we can analyze changes in these networks to see if their social circle is shrinking or growing, and if the strength of their connections is increasing or decreasing over time,” she says.

Ram additionally used the CatCard data to look at the regularity of students’ routines and whether they had fairly established patterns of activity during the school week. She and her collaborators developed a machine learning algorithm to determine ways to quantify these patterns.

Considered together with demographic information and other predictive measures of freshman retention, an analysis of students’ social interactions and routines was able to accurately predict 85 to 90 percent of the freshmen who would not return for a second year at the UA, with those having less-established routines and fewer social interactions most at risk for leaving. 

“Of all the students who drop out at the end of the first year, with our social integration measures, we’re able to do a prediction at the end of the first 12 weeks of the semester with 85 to 90 percent recall,” Ram says. “That means that out of the 2,000 students who drop out, we’re able to identify 1,800 of them.”

Ram found that social integration and routine were stronger predictors than end-of-term grades, one of the traditionally used predictors of freshman retention in higher education.

The problem with relying solely on grades for making predictions is that national literature suggests freshmen who ultimately leave the university make the decision to do so in the first 12 weeks of the 16-week semester, and often as early as the first four weeks — long before final grades are posted, Ram says.

“A public university like ours is very large, and students can get lost,” Ram says. “There are social science theories that indicate that when these students come in, they need to establish a regular routine, learn how to manage their time, and they need to get socially integrated. Those are some of the reasons they tend to drop out — they’re not socially integrated and they haven’t established a regularity of routine on campus.”

Retention: An Age-Old Issue in Higher Ed

Ram’s research represents a new approach to an old problem, and she hopes it can eventually be used by the UA and other universities to supplement the predictive analytics work they are already doing.

“Student retention is something that’s been studied for the last 30 or 40 years, but we never had the ability to track people’s behavior and movement and extract their social integration patterns,” Ram says. “We have also made great strides in developing machine learning and large-scale network analysis methods that help in analyzing such spatiotemporal data.”

Ram’s work comes at a time when universities nationwide, including the UA, are committing more resources to harnessing data analytics in ways that support student success.

“The kind of move that universities are making toward predictive analytics and using more data to understand the student experience allows us to look earlier and more often at some of the variables that we can get our hands on and ask different sorts of questions than we were able to ask before about the freshman experience,” says Vincent J. Del Casino Jr., UA vice president for academic initiatives and student success.

The UA, which saw its freshman retention rate jump from 80.5 to 83.3 percent between 2016 and 2017, has for the past four years contracted with outside vendor Civitas Learning for data analysis related to retention and graduation rates.

The university now uses some 800 data points — related to everything from academic performance to financial aid to students’ activity in the university’s D2L course management system — to identify which students are most at risk for leaving. Those predictions, which do not include data from Ram’s research, are about 73 percent accurate from the first day of classes, with the rate of accuracy improving over time, says Angela Baldasare, UA assistant provost for institutional research.

The university’s current effort to predict first-year student retention is a significant improvement over previous models, which relied largely on descriptive data from surveys and incoming freshmen’s “academic index scores,” predictions of their first-year college GPA based on their high school GPA, the difficulty of the courses they took in high school, and their SAT or ACT test scores.

The university now generates lists, based on those 800 data points, twice a semester and twice in the summer of the 20 percent of students most at risk in each college. Those lists are shared with the colleges with the intent that advisers will use them to reach out to students who may need additional support or guidance, Baldasare says.

“As early as the first day of classes, even for freshmen, these predictive analytics are creating highly accurate indicators that inform what we do to support students in our programs and practice,” she says. “The lists of students are delivered in the fourth week of classes because we know students may already be making up their minds about staying or leaving the UA.”

The university also is preparing to launch an online dashboard where advisers can access key data and assess student risk in real time throughout the semester.

“We’ve really worked hard over last year to work more with advisers and get advisers better data so we can facilitate their informed action with students,” Baldasare says.

Predictive analytics merely provide a signal for underlying challenges students might be facing, Baldasare notes. It’s up to the advisers on the ground to use that information to diagnose the problem and help students as best they can, with the understanding that it will never be possible to retain everyone.

“What all of this ultimately boils down to is individual students and individual needs and how we translate the big signals down into individual outreach,” Baldasare says.