Climate Change’s Big Impact on Big Data

by Taymour | March 12, 2015

The problem inherent in climate change studies is not the amount of data to manage, the technology trying to tame that data, or even the people who use the technology. Science (or more acutely, certain populations’ unwillingness to believe in science) actually poses the greatest challenge to big data. According to the Pew Research Center’s most recent poll, only 40% of Americans believe that global warming is primarily caused by human activities that pump excessive amounts of C02 into the atmosphere. If you belong to this minority, keep reading. If you do not belong to this minority, definitely keep reading. 


Science Skepticism 

Those who understand the scientific community have a sense of how elusive consensus among scientists can be. By its very nature, science is based on debunking science with better science. Thus, to have an overwhelming majority consensus among the world’s scientists is extremely significant. To be sure, ample room is still left for debate. That debate, however, is more focused on the magnitude of climate change’s impact than its origins.

For this article, I will give corporate America the benefit of the doubt by assuming that it is more scientifically inclined than the population at large (allotting for variations based on industry, geography, and other factors). However, this paper isn’t trying to accurately quantify business’s scientific inclination; instead, it’s about the alarming proportion of science naysayers (both active and passive) who undermine data-driven, actionable insights. Based on my own experiences with a variety of businesses, I estimate that about half the people in corporate America fall into this category.

If I’m correct, then one in every two businesspeople in America disregards data regardless of how reasonable or compelling its conclusions are. These people trust their guts more than any scientific consensus. Sadly, they often dismiss reason and empirically-derived conclusions even before considering any analysis. If they do eventually consider such results, asking them to give the matter any subsequent validation or further analysis seems to be asking too much. These trends reflect a different enemy of science: the irrepressible urge for short-term results. The data-driven professional understands that the initial pass at the data may have been incomplete. Hypotheses must be revised. Assumptions must be revisited. New data sources must be considered. Qualitative opinions must be factored in. In other words, science is about continuous improvement in the search for answers. When an answer is found, science immediately attempts to debunk it with whatever methods make sense. If the debunking effort fails, the answer will remain as the best conclusion, at least for the moment. The data scientist will not dismiss an answer just because it is a non-quantifiable, emotional response. Science will accept an answer as the best one only if various means to disprove it have been extensively explored. In this sense, science is a non-judgmental, “show-me-what-you’ve-got” discipline.

Indeed, other non-quantifiable attributes should be considered in lieu of or in addition to the data-driven methodologies. In fact, any conclusion should be treated with skepticism, be it data-driven or not—such is the way of science. However, we must discourage knee-jerk rejections of the empirical approach merely because it doesn’t “feel” right. A blanket policy of skepticism toward testing, profiling, modeling, and other forms of data analysis is not helpful.


Gut Instinct vs. Data in Corporate America

Data can actually be used to validate hunches and predict events using empiric methodologies. In the end, the data-driven approach employs the tried-and-true scientific method in an attempt to augment, if not supersede, what we think is true. Inevitably, some results will seem counterintuitive to conventional corporate wisdom, which is the fun part for data scientists (that is, unless political motivations lurk behind the scenes). Corporate America’s resistance to big data seems to hold its practitioners back from being fully integrated into mainstream corporate functions—a sad state of affairs, given what big data has to offer.

If my estimated 50-50 ratio of science supporters to science naysayers is even remotely accurate, that’s a big problem. Because we can prove whether a particular idea pans out by using a well-designed statistical test, any arguments against taking the actions indicated by the results should be based on data flaws or problems with the methods used to analyze it. They should not be based on an individual’s intuition. 


The Best Challenge to Science is Better Science 

Succinctly put, one should only challenge science with better science. In some cases, the data and what it’s measuring may be mismatched. Or, the science naysayer may argue that some human behavior is just not conducive to measurement. Even so, a dialogue about how to improve data and analytical practices is far more helpful than a blatant rejection of their merit. That dialogue’s merit will depend on how the parties involved value a data-driven, scientific approach. So long as some people continue to resist the broad scientific consensus about climate change, you can be sure data scientists will remain undervalued members of corporate America.