Not Your Regular BI

The catch (all) phrase:

One of the frequently used buzz words tossed about lightly in client meetings and in general when someone’s trying to impress company is “analytics”. The sheer versatility of the term is also why it is least clearly understood – since it could be anywhere in the continuum starting from cleaning and presenting raw data in a form amenable to further number crunching; to canned reports or metrics presented in a glitzy dashboard; to sophisticated predictive statistical models and optimization. Advanced analytics is mostly about the third part of this spectrum. While we’re on analytics, let’s not even venture into the area of Google-based research producing voluminous essays, which also goes under the guise of analytics.

Gartner has listed advanced analytics as number 2 in the Top 10 strategic initiatives for 2010, based on conversations with CIOs, hype cycles and specific interest generated in the market. IDC’s prediction for predictive analytics tools shows a compounded growth of 8 percent over the next 5 years.

The proliferation of boutique firms with highly specialized industry expertise, and the spate of M&A activity, both point to the continued growth prospects of advanced analytics. While in the service oriented segment, there are a few cases showing loss of focus after a niche firm has been acquired by a larger diversified firm, fact remains that in the products space consolidation is the name of the game, leading to bigger and hopefully better advanced analytics products with significant market reach – IBM taking over SPSS should give SAS the big blues, no doubt. Things have clearly moved past the exploration and interest phase for analytics.

What’s the catch?

It is usually difficult for buyers to see through the marketing haze, especially with a myriad of solutions that claim to do everything in the analytics space. It doesn’t help either, that the term analytics is so ambiguously used to cover a huge gamut of solutions.

The need:

The need for each of these does exist, for different firms depending on the maturity of the firm in using data for “competing on analytics”, as Tom Davenport puts it.

Business users need accurate, clean and timely data to take good decisions. This is usually the grunt work that most in-house teams hate, but can’t avoid. They may use the IT department for generating data because the servers are in IT’s custody, but IT may not have the best view of what the business wants. Worse, business users may do it within their own team, wasting the time of expensive resources whose main work is held up because of reporting duties best handled by specialists with data management skills. Without good usable data, the promise of analytics falls flat, so while this is step critical, it is important that the right people for the job do it, so that business users can focus on what they do best.

In better cases, periodic reporting has been automated to a certain extent, but the usability suffers either due to lack of visual appeal or due to security and licensing restrictions. Some product companies have made a killing with potential clients, with visual impact scoring over analytical abilities of their competition. Visual data exploration has an exemplary proponent in Hans Rosling whose TED talks on macro economic data analysis and insights are legendary.

While everyone may not be have Rosling’s gift, the best are already there, going further than interactive graphs, going the whole hog using sophisticated tools to build statistical data crunching models. Banks have been at it for years. So have retailers and insurance companies, not to mention sports franchises, hospitality majors or your friendly neighborhood airlines jacking up the rates at will. The Holy Grail lies in the accuracy of predicting outcomes – be it finding out who will respond to a 10% discount offer; who will win the 2010 Super Bowl; what interest rate John Doe deserves for his new auto loan; or what inventory levels of each SKU should be kept to balance inventory turns and stock-outs.

And that calls for some wizardry, in the form of domain knowledge coupled with a thorough understanding of data management and statistical analysis techniques. If it had been that elementary, dear Watson, everyone would’ve been there.

One size doesn’t fit all

The time is overdue for every firm to do an objective evaluation of its situation, pressing analytical needs that would help solve business issues better, balance the short term and long term goals, and take the right step forward in the direction of advanced analytics. The competitive advantage offered by advanced analytics is officially up for grabs to anyone who is interested, has data and a culture that encourages fact-based decision making.