You Think Your Organization Is Data-driven: Think Again

Are you engaging in unbiased data analytics? Here are strategies for smarter decisions.

April 3, 2023

Your enterprise has invested in myriad analytics tools. You have capable data scientists and analysts on board. You believe in the power of letting your data guide your business. You think you’re making data-driven decisions. Then you see these decisions go awry. Michael Amori, CEO and co-founder at Virtualitics, discusses how to find out if your organization may still be guilty of decision-making by hunch and strategies to remedy that.

The amount and complexity of data that’s now collected and analyzed is too vast for even experienced data professionals to sort through. So, they resort to what’s worked passably well in the past: They create a hypothesis to test.

Further, this educated guesswork is often skewed toward the highest-paid person’s opinion (HIPPO). Foundational questions get formulated around whatever company leadership believes to be a priority. Right out of the chute, you may be trying to answer the wrong questions.

Guess-and-test approaches to data exploration leave out important, unexamined insights. Here’s how to avoid assumptions that take you in the wrong direction and better leverage AI to follow where all your data is actually pointing.

Avoiding Bias in Data Projects

Bias and best guesswork are closely linked. Take the example of HIPPO bias. An executive may be convinced any productivity dips during COVID-19 were due to the workforce not being in the office. Any analysis done at this executive’s request would be narrowed before it’s even underway because the data analytics team keys in first on the exec’s preconceived notion. They may miss factors that are actually more important: the engagement impact monitoring software has on employee productivity, childcare challenges, and remote work technology difficulties. And instead, investigate a very narrow scope to justify a policy requiring a return to in-person work.

The first step in avoiding bias is recognizing much is hidden in the huge universe of data available. The second step is in knowing what we choose to investigate and also injects bias. The solution is to acknowledge humans can’t really begin to figure out what’s going on within modern datasets without the help of AI, used for exploring data objectively up front and drawing our attention to what matters.

Intelligent Exploration Leaves No Data Behind

Think of a first responder examining an unconscious accident victim who can’t say what hurts. The medic looks at obvious trauma, yes, but they don’t stop there. If they did, they might overlook a hidden injury that poses an even bigger threat to the patient.

This is what intelligent data exploration looks like. It allows you to be sure you’re looking at all important contributing factors to a scenario, even when some are far more obvious than others. It also ensures your personal observations aren’t limiting your scope. Back to the first responder example, they don’t limit their assessment to what they have the most experience with or have seen most often. They approach each patient assessment with fresh eyes.

To illustrate with a supply chain example, say a large e-tailer is concerned about delays in shipments of certain goods. There could be hundreds of metrics to consider to discover why some deliveries are on time while others aren’t, everything from shipment and destination locations to weight, package dimensions, type of product, time of year, cost, and more.

A data analyst might think, “I’ll start by looking at origination locations and follow that path; maybe India is underperforming China.” If the hypothesis turns out to be correct to some degree, the analyst might stop there.

But there may be a factor impacting delivery delays that is, in reality, more important. Intelligent exploration provides open-ended inquiry at the front end of the analysis and uses AI, not someone’s best guesses, to make sense of complex datasets and process them into understandable interactive 3D visualizations.

These visualizations are capable of revealing all the connections going on in a dataset. They can incorporate up to a dozen different dimensions of data, all types of data (categorical, numerical, IoT-sensor-based, etc.),  and literally hundreds of millions of individual data points.

This allows the supply chain analyst to discover groups that are experiencing more delays than others, with AI describing the differences between the groups in natural language. All the factors associated with the most delays – packages originating in India that used a certain shipper, were of medium size, weighed between eight and 12 pounds and were destined for Europe—are apparent. And it turns out package size is a bigger factor than shipping origination point.

Decision makers in this example now know to make solutions for larger packages a top priority, not something that a hunch or limited data exploration would have revealed. By looking at all the relationships in the data, they know where to intervene.

See More: How Decision Intelligence Brings Unparalleled Value to Businesses

Removing Bias Is Key for Truly Data-driven Decision-making

There’s a natural inclination to gravitate toward what we already know. But limiting what’s considered at the outset of a data analysis project is risky. It leaves out insights that can incorrectly confirm biases. It can fundamentally change how you understand business problems. Data can appear outsized because other data providing context has been excluded. It can lead you to act in the wrong direction.

Intelligent exploration is the antidote to guess-and-test. It leverages artificial intelligence not to supersede an analyst or SME, or CEO but to lead them to consider new ideas and challenge assumptions. With more than 2.5 quintillion bytes of dataOpens a new window  being generated each day, this automation is the only way to make sense of data in all its complexity—and achieve true DDDM.

How are you improving your data-driven decision-making processes? Share with us on FacebookOpens a new window , TwitterOpens a new window , and LinkedInOpens a new window . We’d love to know!

Image Source: Shutterstock

MORE ON DATA-DRIVEN DECISION-MAKING

Michael Amori
Michael is the CEO and Co-Founder of Virtualitics. Michael is a data scientist and entrepreneur with a background in finance and physics. He co-authored many of the patents underpinning the Virtualitics AI platform. He believes that AI applied to data analytics can help solve some of the world’s toughest challenges. Prior to co-founding Virtualitics, he was a Managing Director at Deutsche Bank, where he started and headed a data driven trading desk in London, New York, and India focused on the insurance and pensions markets. Managed a large group of data scientists. Designed the group’s predictive modeling and risk management systems. Started and was technical advisor to the bank’s first insurance-linked fund. Previously was an interest rates derivatives trader at various large banks. Graduated from the Goldman Sachs analyst program. Michael was also a researcher in nanotechnology at Caltech, where he contributed to two papers and obtained one patent. The patent is related to a quantitative method for determination of target molecules in the context of nanotechnology devices applied to medical research. Michael is interested in science education, especially for early childhood, to reduce inequality. He is a member of the Board of Caltech’s Computer Science Department (IST division), a member of the Board of the Pasadena Math Academy, and judge at the LA County Junior High Physics Science Fair. He was a member of Caltech Children’s Center Board. He has an MBA from Harvard, an MS in Applied Physics from Caltech, and a BS in Physics with honors from Columbia University.
Take me to Community
Do you still have questions? Head over to the Spiceworks Community to find answers.