Top Resources for Analysts: Excel, Data Analysis, and Business Intelligence is another list you shouldn’t miss.

If you work with data for a living, the following sites are worth a visit (or a subscription) to learn from some of the best, most passionate practitioners.

Data Mining, Analytics and BI roll-outs are unlike any other project your organization may have undertaken. Political and non-technical issues can derail the fragile project which is anyway struggling to handle ambiguous and constantly changing requirements.

Mark Demarest, in his classic 1997 article The Politics of Data Warehousing says:

Data warehousing projects are frequently side-tracked or derailed completely by non-technical factors, in particular the political treaty lines within the firm, and the politicized nature of data itself. Because data warehouses are infrastructure for sociotechnical systems (STSs) within the firm, politics and the exercise of power are inherent in data warehousing projects, and data warehouse designers have to adopt work practices and methods from non-technical disciplines, think of themselves in new ways, and employ some fairly sophisticated qualitatively sociological methods in order to optimize the chances for successful deployment of data warehouses.

The Politics of Data Warehousing also lists down 10 warning signals to detect politicization of the project, and 10 countermeasures – essentially the recipe for delivering on the project promise.

Larry Greenfield, in a more recent article titled Data Warehousing Political Issues identifies three common threads as:

1) Data warehousing imposes new obligations whose responsibilities are unclear
2) Data warehousing requires changes in processes that an organization is comfortable with
3) Data warehousing requires agreement on some, but not all, definitions of data.

Larry classifies the political issues into those that are within the IS organization (IS to IS), those that are between IS and the users (IS to Users), and those that are between users (User to User). Click here to read Data Warehousing Political Issues.

And don’t miss the article Data Warehousing Gotchas, listing some little known nuggets of wisdom and experience that can save your project. As Larry says, “Forewarned is forearmed!” Click here to read Data Warehousing Gotchas

Any innovative/disruptive technolgy has first gone through a phase during which the technology is made simple and more useful for its end users. Unfortunately, BI vendors are still focussed on adding more complex features (that even their own teams don’t know how to use) instead of focusing on simplicity and productivity. And the third-party consultants sustain the con because it helps them hike their billings.

In the short run, the client suffers. But in the long run, the analytics profession suffers as a whole due to unmet promises and millions of dollars in sunk costs.

The way out? Simplicity. Read the DMReview article below for a ’simple’ way of reaping huge BI benefits.

A Simple Plan: Embracing Simplicity Can Reap Huge Business Intelligence Benefits

The true value of BI is in viewing (and nurturing) it as the cognitive base and a response model for an organization in dialog with the ‘external reality’. In that sense BI is a truly disruptive phenomenon, even though it is the logical next step for the all-round ‘digitization’ that has been taking place for the last few years.

Yet, most managers appear to be struggling to fit Analytics into their traditional repertoire instead of looking at it afresh and leveraging it like any disruptive technology should.

An interesting HBS Working Knowledge article about Disruptive Innovation that may give you some ideas: Six Keys to Building New Markets by Unleashing Disruptive Innovation : HBS Working Knowledge

Also interesting is the classification of the system environment into Rigid, Deterministic, Purposive, Heyristic, and Purpose Seeking, and the imapct of this classification on BI design.

Click here to read the article ‘A Systems Theory of Business Intelligence’.

The paper is targeted at financial services that are not doing analytics yet (there are quite a few of them around actually!). It sets out a road map for doing analytics by starting small to quickly start delivering business value, and then gradually taking on more complex projects.

Click here to read Delivering on the BI Promise: A Financial Services Road Map

Thanks to Dratz for letting me lift a graphic from his paper on BI as an Abstraction Architecture.

Business Intelligence: Not Just for Bosses Anymore – Editorial – CIO

Having worked with decision support software since the mid 1970s, both as a user and as a vendor, I had been very disappointed by the shallowness of the coverage of OLAP by industry analysts. Most had never even used an OLAP tool and they seemed to be more concerned with checklists of often-irrelevant, vendor-promoted features than how products really worked and what they were capable of doing for users. It seemed to me that this was the equivalent of a motoring magazine written by non-drivers, and no more useful. I was also concerned that vendor-sponsored ‘research’ documents seemed to be too prevalent then, as now.

Unlike most other information resources, the OLAP Report consciously maintains its independence by staying away from any kind of vendor sponsorship and advertisements. It is entirely funded through subscriptions, and if you are buying it you will find it worth its price.

A not-to-be-missed article on The OLAP Report website is “How not to buy an OLAP Product” that offers excellent advice on the planning and purchase of an OLAP solution.

The consequences of a bad decision either mean millions sunk into a complete failure, or even more dangerously: the project being declared a “success” due to political compulsions without any tangible business benefits. The latter is a fairly common scenario because heads would roll if anyone were to acknowledge a multi-million dollar goof up. So the farce goes on, and the vendor keeps earning their annual licensing revenues, just because the CxO is too scared to acknowledge that the king is naked.

The keys to a successful OLAP evaluation are:

• Evidence-based selection criteria: You need to have some measurable and demonstrable criteria for making the selection.
• An understanding of user needs (instead of OLAP as a fashion statement)
• Managing vendor distractions: You cannot let the vendor drive the evaluation and tell you what your selection criteria ought to be. “Projects where the vendors’ excellent sales efforts were a major factor in product selection were among those that achieved the least in business terms.”
• Checking Reference Sites: Not all the “references” may be happy/satisfied customers. Some actually are extremely unhappy, just that they cannot say it without looking dumb and losing their jobs
• Understanding Financial & Licensing Issues: “Remember that OLAP software license fees are only a small proportion of the total costs of a project.” Yes, commercial OLAP TCO goes into multi-million dollars.
• Bake-off or Proof of concept: Extremely important for comparing shortlisted vendors on a small project. But make sure that the POC does not get just patched up into the final system!!

If you are planning to implement a commercial OLAP solution, don’t forget to read The OLAP Report: How not to buy an OLAP product.

Or you can get an open source OLAP at a ridiculously low ownership cost. (See posts Open Source OLAP – Part I, and Part II)

Need help? Drop a comment and we could do something together.

Margaret Dunham, the author of Data Mining: Introductory and Advanced Topics once said:

Data mining research and practice is in a state similar to that of databases in the 1960s.

That is true, but will have to change. Databases have long been standardized, and data-mining & BI would get standardized in due course. Till then, I guess, we’ll have to do with whatever information is available, ignore the hype, and take the plunge for defining our own solutions.

So what does it take to deliver upon the BI promise? Data, for sure. And then you need a lot of meta data as well to make sense out of the data. And most importantly, the context for analysis – the what, why, and how of the effort. In today’s implementations, the context is implicitly identified and built into the solution, which may not be flexible enough to meet the changing needs of the users.

As an excellent effort in the right direction, Dratz recently published a white-paper outlining why BI should be viewed as an Abstraction Architecture for Information, and how it should be structured. And why a flexible BI architecure should have be the ability to apply dynamic contexts at run-time for the conversion of data into meaning. He proposes a very interesting piece therein: a Transaction Clearinghouse that would house the data till all pieces of an enterprise transaction (possibly spanning across multiple systems) are available, and would apply contexts dynamically to create meaning out of the data.

Even if you aren’t sure what all this means, do read Introduction to an Abstraction Architecture for BI white paper by Dratz. It will give you a very good overview of what BI is, why it is needed, and how the vision could be actualized.

In case you’d like to first read a little bit of background about the white paper, you can read his post Abstraction Architectures…or, Why Everyone Should be thinking of BI.

Do let me know what you think?