It’s football season, and if you watch games, you’ve probably seen something like this. It’s fourth down, deep in your opponent's territory. Should you go for it? Should you attempt a field goal? How do you decide when there seem to be strong arguments for taking one of several actions? Over the past few years, it’s become more common for the broadcasters to throw up a graphic on the screen, and to conclude “analytics says” to choose one or the other of the available options. If the team chooses to do what the analytics say, and that choice seems unusual or it flies in the face of past practice or received wisdom, and the outcome isn’t good, the coach or team can expect to be excoriated by fans and analysts alike for their bad decision.
Unless you’ve been living under a rock for the past couple of decades, or you have absolutely no interest in sports, you’re probably well aware of the rise of analytics in sports. Moneyball (the movie), and before that Moneyball (the book), certainly helped increase awareness. Whether they increased understanding seems like a different question.
We’re old enough to remember a time when you didn’t have all manner of sports statistics available instantaneously, when to get some facts and figures you’d have to read box scores in a newspaper, or even peruse the backs of player trading cards. (Notice we didn’t say we’re exactly nostalgic for that time.) But the point remains that there has always been plenty of data available, and someone to compile it into various statistics.
As data people, we obviously want to see organizations utilize data when they make decisions. But it’s not just a matter of using data. Think about all the data available to sports franchises across all sports, much of it dating back decades; it still wasn’t until analytics became more advanced that they realized that some skills were over- or under-valued, that some cherished statistics really bore very little correlation to team success, and that a number of routine in-game decisions were frequently suboptimal strategies. (It turns out, for example, that being more aggressive on fourth down on balance helps teams score more points, and thus increases their likelihood of winning games.)
The thing we always wonder when we see the “Analytics Say” graphic is, what’s the calculation? Not necessarily what is the math involved, but how is this conclusion reached? And by how much is the analytics-recommended option superior to the other ones?
We prefer to believe that major sports broadcasters don’t display this information unless they think their model is well trained on robust data.
And having a good model trained on trustworthy data is critical for businesses and organizations of all kinds that want to leverage their data. Of course, it may be easier in business than in sports fandom to see the value of marginal improvements, whether that’s in increased revenue, decreased expense, or even something less quantifiable for the bottom line.
And it may be easier in business than in sports fandom to be dispassionate about the decision, although we’ve seen plenty of situations where a negative business outcome casts retroactive doubt on what was a data-driven decision process. As a business, if we decide to end a program, or redirect some resources away from it, and it’s one I’ve invested time and labor in, I may question or even actively resist the data and the analysis that recommends that course of action.
Still, it seems to us that sports fans are by and large comfortable with data, and can use statistics to make and support arguments about which player is superior, or which coach makes better decisions, or which team is more likely to win. So decrying the use of data and statistical analysis (by blaming the “nerds”) in some situations does contain a certain irony.
It’s common now to bemoan the lack of data literacy in the workforce, and to attribute at least some of an organization's continuing inability to be data-enabled to this lack of employee skill. But we suspect that many of these same employees are sports fans, and that they can make fairly sophisticated arguments for and against game strategies, and that they track numbers quite closely in their fantasy leagues or gambling applications. Are they statisticians or data scientists? Well, maybe some of them. But do they have sufficient quantitative skills to perform comparative calculations, to engage in some what-if scenarios, and to understand simple charts and graphs?
Many of these fans probably got familiar with sports statistics at a relatively young age, and they pay regular attention to them. The challenge for businesses seeking to leverage data might be more around make their internal statistics just as easy to find, understand, and utilize as what is available on fantasy sports sites!
Despite the fact that much more data is available to decisionmakers, in multiple formats, too many organizations haven’t really improved their decision processes. We have seen many organizations just throw more data, more data products (such as dashboards), and more data tools at the problem. In a vacuum, more data may be better than less data. In real life, we have observed that, well, better data is best.
What are the underlying issues that prevent organizations from getting to this place where they have useful, structured, and available data. We noted data literacy as an ongoing challenge a few paragraphs back. Data silos continue to plague us, and data quality becomes even more difficult to maintain in the face of new systems and technologies. Data security policies and practices have trouble keeping up.
We see all of these issues as being things that improved data governance will help address. Now, we typically recommend that you pick a high priority data challenge or two and address them first. It’s probably a fool’s errand to try to upskill workers, stamp out silos and shadow systems, solve data quality, update and enforce security and access regulations, and track data lineage; all the while continuing to churn out data products of questionable provenance and uncertain value. Those are all valuable efforts, but tackling them all at once probably means succeeding at none of them.
We know it’s a wild suggestion, but maybe you should start by making sure everyone in the organization knows why the percentages favor going for it on fourth and short, so to speak. There’s probably terminology in use at your organization that not everyone understands the same way. It’s virtually certain that the metrics used to measure success are not understood the same way across all units, and we suspect it’s likely that in some cases there is no understanding at all.
Not all models are created equal, and not all data is equally valuable. New metrics, even if (maybe especially if) they have superior explanatory or predictive power, need to be explained, their utility must be demonstrated (over and over), and they have to withstand challenges from traditional metrics as well as other new metrics, in order to be recognized and put to use.
In our football example, we assume that the analytics in this situation take into account the following factors, among others. How deep are you into your opponent’s territory? How many yards to go to get a first down (or a touchdown)? How much time is left in the game? What’s the score? What is the likelihood of success or failure for each option, and where does each of those outcomes leave you?
Having observed this trend over the years, we speculate with some degree of confidence that these models refer back to similar situations in thousands of previous games and extrapolate from them the various values (whether it’s scoring more points, or having a greater likelihood of winning) of the potential outcomes, then multiplying those by the probability of the outcome, and then comparing the results. And we further speculate that the difference is often pretty small: maybe the analytics say kicking gives you a 46% chance of winning, while going for it gives you a 47% chance.
When we talk about business strategy, and decision making, and using data and analytics to support them, aren’t we in a similar situation?
If you go to your organization’s leaders with a document or a presentation that says, "the analytics suggest" taking a certain course of action, they will probably want to know how you came to that conclusion. A certain amount of quantitative and potentially statistical explanation might be required, but as long as your organization has an agreed-on terminology around data, you should be able to complete this explanation and have it understood. A business glossary, where data-related business terms are defined by the people responsible for the data, is invaluable here.
Ideally, these people will have follow-up questions about what the summarized or aggregated data looks like, where the data came from, whether there’s other data that might be useful as part of the analysis, etc. All of these artifacts can be part of a data catalog, which can include an inventory of data products and assets, documentation about integration and lineage, and a place to look for and from which potentially to explore other data sets.
When the analytics don’t make sense, or the dashboards seem off, or there are questions about the underlying data, a place to submit data quality concerns and review existing quality rules can help.
Our Data Cookbook integrates all these features together, forming a one-stop shop or a kind of data help desk. When you use it in a spirit of pragmatism, encouraging users to ask, answer, and review questions about data, we believe it helps build confidence in data—confidence that it is accurate, confidence that it is useful, confidence that it can be understood—and data literacy together. With the Data Cookbook as the centerpiece of your data governance, data intelligence, and data catalog efforts, “the analytics say go for it” isn’t just the answer to the question, it’s also the beginning of who knows how many valuable and productive conversations.
Hope you found this blog post beneficial. To access other resources (blog posts, videos, and recorded webinars) about data governance feel free to check out our data governance resources page.
IData has a solution, the Data Cookbook, that can aid the employees and the organization in its data governance and data intelligence efforts including data quality. IData also has experts that can assist with data governance, reporting, integration and other technology services on an as needed basis. Feel free to contact us and let us know how we can assist.
(Image Credit: StockSnap_sports-ball_HWDNFMZHRP_Football_DataGoal_Bp #B1277)
