This is the first in a series of short blog posts where we explore common varieties of bias that can beset analytics projects. Bias has serious ramifications for the success of analytics in any organization. Understanding the nature of bias is crucial for understanding the extent of a model’s accuracy. In this first post, we discuss what bias is, why it occurs, and why it matters (a lot).

I’ve lived through this phenomenon first hand. The environment was new to me, sitting at my assigned seat at the cherry wood conference table for the weekly executive staff meeting. I was told very clearly that I was to stick to the presentation, answer only when asked a direct question, and never, no matter what happens, ask why! After being ushered out of the meeting when I finished, we quickly huddled for a post-meeting debrief. Everyone started asking “How do you think it went?” “What do you think he meant when he said this?” and “Did you understand what he asked us to do?” I finally asked, “Why didn’t we just ask him?”

There is growing literature around interpretable machine learning and explaining black box outputs to humans who will make real decisions based on the results. Predictive model interpretability is a nuanced and complex subject. For example, AlphaGO, the experimental deep learning solution created by Google to play the ancient board game Go, made headlines recently for defeating a Go grandmaster for the first time. This was a significant milestone for a machine learning system since Go is significantly more complex than chess.  When Go masters took an interest in AlphaGO’s winning strategy, the program’s creators faced a familiar question: Why did it choose certain moves?

Data science and predictive analytics’ explosive popularity promises meteoric value, but a common misapplication readily backfires. The number crunching only delivers if a fundamental – yet often omitted – fail-safe is applied.

There are two problems with humans making decisions from data. We are biased— even experts are just as likely to give inconsistent judgments—and we don’t always understand, or trust, the model. Although decision-makers could benefit from using data as a part of their decision making, raw machine learning results may not be meaningful enough.  So how can we use data in a way that experts trust without diluting the machine learning process?