Companies with large analytics budgets often create the problem of too much data. Teams become so eager to impress others about how much data they have and how fantastic their reports and dashboards are that they end up generating thousands of metrics and an excessive number of descriptive and predictive models that together create negligible value.
Of course, under many circumstances, the right data lowers risks and improves decision-making:
If you offer a subscription service, usage data can give you early warning about inactive customers so you can take quick action to avoid losing their business at renewal time.
If you use process automation tools, activity data lets you create a closed feedback loop to improve decision points and increase process efficiency.
If you provide services to disadvantaged job seekers, data about outcomes (% of participants who get a job and increase their income) can help you direct resources to where they can produce the highest return and use your positive community impact to attract more funders.
Yet, statistical analysis can only take us so far. When we're talking about creative work or true innovation that leads to above-market results, data can also blind us to important facts that exist outside our model.
Historical data is not going to tell us whether an unusual marketing campaign will incite emotion and stand out, a new business model will take off, or a novel intervention to reduce criminality among underprivileged youth will succeed.
Above-market results are rarely found in incremental progress based on quantifiable data
Many opportunities can only be exploited if we ignore past data and rational economics thinking.
In 1999, Tony Hsieh almost deleted the voicemail he received from an even younger entrepreneur, Nick Swinmurn, who had an idea no investor would touch: selling shoes on the Internet. Zappos went from an unpopular idea to a $1.2 Billion business in just 10 years.
In 2001, when Apple announced it was going to open the first Apple stores, many publications and pundits predicted that the concept would fail. Fifteen years later, data would show Apple ranking as the top retailer in sales per square foot, with more than 400 stores worldwide.
Also in 2001, the Becoming A Man (BAM) program was launched in Chicago to help young men navigate difficult circumstances that threaten their future. Program founder Anthony Ramirez-Di Vittorio adopted an unusual approach at a high school to create a safe space for young men to receive support and develop the social and emotional skills they needed to succeed. In 2009, BAM won the Crime Lab's "innovation challenge" and received funding to scale up the program to 18 schools. 20 years later, it has been expanded to Boston, Los Angeles, and Seattle, serving hundreds of schools and thousands of students.
In 2010, Trainmore opened the Netherlands with a novel business model: you pay less as you exercise more. There is a basic monthly subscription, but you save 1 euro with each workout. Because of its unique concept, at first TrainMore struggled to get investors as there was no hard data that could be used to prove its model could work. After it managed to grow from just one to nine locations, it no longer had issues attracting investors.
The people deciding to make those investments knew they couldn't rely on data and analytics to "prove" their ideas had merit. They were willing to "stick their necks out" and take a leap of faith for the sake of innovation.
There is a time to leap, and a time to measure
Most great successes are like the ones described above. They start with qualitative, not quantitative data, and leverage small bets and trials that build on each other. But that doesn't mean we can ignore data. In all those examples, data provided granular insight that helped the organization navigate toward success based on an increasingly accurate map.
In the case of the BAM program, while the initial interest came from anecdotal feedback, the program continued to be shaped based on trial and error, and was later tested twice via randomized control trial (RCT), each with more than 2,000 youth (details available as a working paper in the National Bureau of Economic Research).