We Have More Data Than Ever. Why Aren’t Decisions Easier?

Higher education has never had more data available. We have data warehouses, dashboards, surveys, benchmarking, predictive models, and increasingly AI. We in theory have open government data, although in Canada we also know that much of it is woefully out of date for anything resembling a timely decision.

We’ve spent years making data more accessible and putting it directly into the hands of decision-makers. Yet important decisions don’t seem to have become proportionally easier.

If we have more data than ever, why aren’t we more confident in the decisions we make with it?

Consider something that seems simple: a course waitlist.

We know exactly how many students are waiting. We know who they are, what program they’re in, how long they’ve been waiting, and probably a dozen other things about them.

But what does a waitlist of 50 actually mean?

Do all 50 students genuinely want the course? Is it their first choice or a backup? Would they accept a seat if one was offered tomorrow? Does the waitlist indicate enough unmet demand to justify another section?

Or perhaps the opposite problem exists. Maybe students know the waitlist is already enormous and don’t bother adding themselves to it. Perhaps our institution caps how many waitlists students can be on. Our perfectly accurate count of 50 could actually understate demand.

Suddenly, a question that seemed like it should have a data-driven answer becomes much harder. The data can be completely accurate while still leaving the decision unclear.

From Trusting People to Trusting Systems

For a long time, we dealt with some of this uncertainty through people. Senior leaders received important information from analysts they knew and trusted. Those analysts didn’t just provide a number. They provided context almost invisibly: here’s where this number came from, here’s what changed, here’s the limitation you should know about, and here’s what I think matters.

In some ways, the analyst was part of the data architecture. But in my view, that model is changing: increasingly, numbers arrive through automated systems, dashboards, predictive models, and now AI. A leader may encounter an important number without ever talking to the person who calculated it. In many cases, the number was calculated by a system and there may not even be a person who calculated it, in the traditional sense.

That means trust has to extend beyond the person to the systems, definitions, processes, and governance producing the information.

Some would argue that this shift diminishes the role of analytics expertise in Institutional Research, IT, Finance, and elsewhere. I think the opposite: as my expertise with AI and advanced analytics automation gets stronger, I think human judgement and deciphering what actually matters is becoming increasingly important. 

Access Isn’t Understanding

For years, one of the dominant ideas in analytics was self-service. If we could just get the data out of spreadsheets and institutional silos and into good dashboards, decision-makers could answer more of their own questions.

But just because we gave someone a dashboard doesn’t mean we made the decision any easier. A dashboard might tell a Dean that applications to one of their programs are down 18%. It can show them five years of history, break the decline down by domestic and international students, compare the program with others, and let them filter the results six different ways.

But the Dean’s question probably isn’t “Are applications down 18%?”

It’s more likely: “Do we have a problem, and what should we do about it?”

Answering that requires context the dashboard may not contain. What is happening at competing institutions? Has the applicant pool itself changed? Has government policy changed? Is this a one-year anomaly or the beginning of a longer-term shift? Is there something about the program that needs to change?

Dashboards help, but they don’t make decisions for you. Sometimes they even result in decision paralysis as many analysts try to make sure all information is available, rather than just the most salient parts. Providing additional information is useful, but it doesn’t give someone the experience to interpret it, the context to understand it, or the confidence to act on it. After the last two years, I think we’ve all become more familiar with the uncomfortable reality that sometimes there simply isn’t enough information to remove the uncertainty.

That matters because higher education decisions rarely exist neatly within the boundaries of one dataset. Should we grow a program? Change it? Close it? Add another section? Increase an enrolment target? These questions combine student demand, finances, academic priorities, capacity, government policy, labour markets, and things we simply don’t know yet, or don’t measure.

At some point, someone still has to exercise judgment.

AI Makes Human Judgment More Important 

AI is going to make this problem more interesting. We’re rapidly getting better at asking a system a question and getting an answer without needing to know which dashboard to open, which report to run, or perhaps even which dataset contains the answer.

I think that’s an opportunity (and honestly is the most fun I’ve had with software in 20 years). But it also removes another layer between the decision-maker and the underlying data. If I ask an AI system why enrolment declined and it gives me a plausible explanation in ten seconds, how do I know whether I should trust it? Remember all those times that data governance seemed like a nice-to-have…?

As analysis becomes easier to produce, the harder part may be knowing which questions to ask, what to trust, what is missing, and ultimately what to do next. Put another way, more data and better tools make the human side of analytics more important. 

Introducing Smart Data: The Decision Advantage

These are questions that we’ve been thinking about a lot at Plaid Analytics. And through our work with Higher Education Strategy Associates, we’ve discovered they’re questions HESA has been thinking about too.

So today, we’re launching something new together: Smart Data: The Decision Advantage.

Smart Data is a nine-part professional learning and collaboration series about using data to make better decisions in higher education. It isn’t about learning another analytics tool. Instead, we’re starting with the capabilities that sit around the tools: asking better questions, understanding and governing data, interpreting evidence, telling compelling stories with it, and using it to navigate real institutional decisions.

We’ll also get into some of the difficult decisions themselves, including enrolment, program sustainability, finances, and the growing role of AI.

The series brings together two perspectives that I think complement each other well. HESA spends a lot of its time thinking about higher education strategy, policy and the decisions facing institutional leaders. At Plaid, we spend much of our time thinking about the data, analytics and processes institutions use to support those decisions.

Higher education doesn’t need another reminder that data is important. We’ve spent decades collecting more of it and building better ways to access it.

The next challenge is getting better at turning all that data into understanding and turning understanding into decisions.