Sun 25 Dec 2005
Merry Christmas folks. Hope you have been having a great time.
:-)
Prompted by a reader comment, this post is about that elusive difference between Analytics and regular IT. Or is there really a difference?
You’d remember post about The Technologist’s Manifesto . JC, who has a great blog on Free Web Seminars about Business Intelligence and Data Warehousing) asked:
As a business systerms analyst caught between IT and business, how would you classify business intelligence/analytics?
IT and Analytics:
At some level of abstraction I believe that analytics is the logical and implicit next step to what the technologists have been doing for sometime now: laying down systems that “improve physical operations” and “reduce logical operations”.
The ‘IT revolution’ so far has been about deploying operational systems that record, process, and streamline transactions, and hence reduce the pain and inherent errors of manual processes. Analytics then is the next level of optimization wherein you gradually add decision making capabilities and intelligence to the operational systems (and manual processes as well) to increasingly automate the routine decisions being taken in organizations, and facilitate analysis of problems that were intractable yesterday. In that sense the work that has been done so far was to lay down the infrastructure for a more ‘analytic’ way of doing business in the future.
If historical IT was about optimization through process automation, Analytics is about building on top of that infrastructure towards decision optimizations. As a bad analogy, the difference between ‘regular’ IT and analytics is similar to the difference between the technology used by Wright brothers to build the first aircrafts, and today’s fly-by-wire technology. Both are part of the same progression towards the same objective, one leading to the other. Not IT vs. Analytics, but IT and Analytics.
P.S.: I am a little curious aout dratz’s take on the question, (jc has posted the question there as well) and will keep you updated.
January 25th, 2006 at 12:28 am
I agree that an evolutionary link exists between IT and analytics. My comment is on the gap that currently exists between these species. In my experience, the issue is primarily one of mind-set: a good IT organization is focused on flawless delivery, sticking to schedules, and production environments. Analytics teams, in my opinion, need to be snugged-up with the business, nimble, and comfortable with “directionally correct.” This clash of cultures can make it difficult to communicate or transition between teams.
January 25th, 2006 at 6:41 am
Well, here’s my take. Sorry it’s taken me a while, but I’ve enjoyed the holidays and really stretched it out.
As an IT/BI Architect, I see it as the difference between architecture and city planning. Architects design/build structures that are well designed to do something. City planners figure out not only which buildings should go where, but what (city/enterprise)-wide resources are required to maintain them. They analyze traffic and figure out how to get people where they need to go. They make the buildings relevant beyond their four walls.
In IT, if it were enough to simply digitize pen and paper transactions, we could have stopped after decent document scanners were built. IT (eventually) deconstructs the paper-based process to its component steps and asks “do we need all these steps, or is there a better way?” IT makes operational/tactical/day-to-day proccesses more efficient, scalable, and managable.
Analytics is primarily a strategic tool, a questioning tool. Questions are asked of someone or something. In the case of “Analytics” the same question is asked of someone (business) AND something (IT). Analytics finds the correlated equivalence between the answers (data, one way or another) and massages those answers into results (information).
But it’s a mistake to rank them (IT & A) or to assume that one is less dependent on the business side than the other. It is also a mistake to not keep IT and Analytics tightly coupled, as akward as that can be.
Analytics makes IT better; IT makes better Analytics. Analytics done right should make IT much easier to do by improving scope and sponsorship among other things.
The reason it is difficult to traverse that wall between the two sides is primarily arrogance on each side. To be crude: IT eleminates jobs; Analytics eliminates IT jobs. That’s not what they’re necessarily designed to do, but it’s a logical and appreciated side affect (until it happens to you).
IT people love those presentations where they show how much could be gained by changing/automating certain processes and rebuking (in a polite way) any users who try to defend their staff and budget. But they get real pissed off when a BI specialist gives a presentation exposing data quality issues in source systems, demonstrates real usage patterns, and shows how much money was wasted on IT projects/software that really haven’t added any value to the business.
Analytics people can be insensitive to IT and miss on a lot of opportunities because they do not foster a true partnership with IT.
Anaylitics should be the biggest advocate for IT because it is what proves the value of IT. And good analytics are dependent on good source systems. For success, there has to be a partnership.
In a good shop, BI/Analytics is brought in before project managers. PM’s can just respond to business requests; Analytics can help evaluate the validity of business requests. But Analytics is a component of IT, not an evolution away from it.
Ok, this comment is getting a lot longer than it should be and I would prefer to expand on just about every point, but I shouldn’t. I’m not sure I did much more than ramble on a bit, but it’s just my 2 cents having just read the thread.
At least you get conversations going, well done.
January 27th, 2006 at 3:32 am
I wholeheartedly agree with the need for Analytics and IT to be in partnership. Its just so hard to do that I’m looking for practical advice on how to bridge this gap.
In my experience, the awkwardness of this coupling isn’t so much a product of fear but of misunderstanding. Coming from the Analytics side, I’ve been frustrated by common IT instincts to:
* Demand comprehensive list of requirements for every request
* Focus on the technical challenges, not the business problem at hand
* Move slowly even for the smallest of requests.
These behaviors make it difficult for an analyst to ask new questions and rapidly provide answers — necessities if the analyst wants to influence decisions with data.
January 27th, 2006 at 9:26 am
You’re absolutely right. And that’s why I say the burden (initially) is on the Analytics side. IT can happily exist without Analytics, but without IT there is nothing to do anayltics on.
The thing to remember is that BI(Analytics) is quite often something imposed on IT and is controlled by a small number of technologists and analysts with very high visibility.
Let me try to offer my advice on the observations you expressed:
* Demand comprehensive list of requirements for every request– There are two reasons for this. One is that they don’t want to promise anything that they are going to be held accountable for if they are not sure they can deliver (this often, but not always, comes in the form of capacity/performance where an analyst defends himself by saying “the IT department said they could handle it.”) The other reason is they are invoking Technology Intimidation (TI) to show you who’s who (and this is usually deployed against people who are technically weak or obnoxious/cocky in the way they approach IT).
To deal with this, remember that even though you have users, IT considers you a user (or at least a consumer). This is not bad, IT people do it to IT people all the time. But you’re certainly not going to get any preferential treatment because you’re in Analytics. And don’t waste anyone’s time. What do you want from them? If you have a specific example, it would be easier to help. Good technologists worry not only about your basic requests, but how security will be enforced, how extraction and storage will be handled etc., in essence all the things required to make it production ready.
* Focus on the technical challenges, not the business problem at hand. –IT solves business problems BY focusing on the technical challenges. While it would be ideal for IT to actually generate business ideas and opportunities for exploitation, they usually don’r have the bandwidth and rarely even know how the business really works. [Why would anyone waste time on a project that does not enhance/support the business? Don’t assume that just because they’re not working on your business problem that their not working on another business problem.]Does your request leverage work already done or does it impose new maintenance (etc.) requirements? How does what you are asking them for fit into their objectives for the year? If you can show them how helping you meets one of their defined goals (or even replaces one), it will be easier to get their enthusiastic support. Are you dealing with technologists or with technology managers? How do you escalate your concerns?
* Move slowly even for the smallest of requests. –This can either be a matter of prioritization or a matter of spite. In either case, you are dependent on them. What are the circumstances under which you engage them? Do you have a kind of step-by-step plan that they can follow and anticipate, or is a along the lines of you need something, then one more thing, then one more thing…? And, what are the implications of your small request? Have you done an impact analysis (they will have to do one)? Does your small request mean a significant reworking to implement or is it a plug-and-play addition (from a technology POV)?
When you give a presenation of the prototype you’ve built, do you invite the technologists who were involved and give them lot’s of credit? Do you send emails to their managers and CC the technologists and your sponsors in praise of the technologists who were so vital to your project? In short, do you make IT feel like a crucial and appreciated part of the project?
If you have an IT department that has plenty of free time to take on extra work, they have a lot of problems that you can’t address alone anyway. If they don’t have a lot of free time, you have to make sure your project is accepted as a project along with schedules, deliverables, everything– or you had better develop personal (not meaning outside the office) relationships with the technologists so they are willing to help you.
Another very important component is your knowledge and at least basic understanding of the technology and systems in use at your company (this is vital to your success). Do you know all the databases and applications in use? Do you know why those systems are being used? Do you know HOW those systems are being used? If you understand the technology stack used at your company, not only will it be easier for you to communicate with IT, it will enable you to answer a lot of your own questions, provide IT with much more specific questions, and even suggest things to your executive users that they may not have considered or known was possible. Don’t be trapped by your executives’ ability to (mis)understand the world of possibilities.
Or you may have a real stinker of an IT department that simply refuses to work with you. If so, you need to get them managed up or out.
Hope some of this helps.
January 29th, 2006 at 1:06 am
As an IT guy who gradually transitioned into analytics, I have seen that invisible wall from both sides. My role in analytics started when I was asked one morning to take over the analytics initiative at a bank when the project manager who was to handle it disappeared 3 days before the project kickoff. We had committed ourselves to a pathetically bad vendor (more on that in my post “The sad state of analytics consulting”) and we never managed to get anywhere close to where I would have liked to take the project. Even though the project was considered a success, it has always rankled me as a personal failure.
A few months back, I struck off on my own and started an analytics products (yes, we’ve been at work putting together an open-source stack) and services (yes, that’s what’s been keeping me away from the blog for the last month) company. And foremost on my agenda was to promote that collaborative IT AND Analytics approach to deliver real value to real people in a real timeframe. I assumed that with my IT background I would be able to empathise with (and get the buy-in from) IT. It wasn’t that easy though.
Inspite of my best intentions, I noticed that even the very senior IT guys at a client were deeply threatened by our presence and felt we were continuously stepping on their toes. They felt that DataWarehousing was a transgression (after all, if a project involves a database, IT should be doing it), or maybe the fear stemmed from the fact that they have historically been reporting wrong numbers. However I think the biggest reason was a blind spot shared by all techies: “If I don’t know about technology X, it must be worthless!”
Another thing I’ve noticed is IT’s predominant “product mentality”. While it is IT’s job to make sure that any delivery is production grade, it does not necessarily have to be a “system”. It would be so much easier to treat many of the requests as “service requests”. Thousands of valid business requests in an organization never get handled because the IT guy is trying to construct a do-it-all reporting system, when all that the business wanted was an ad-hoc one time report (in other words, a simple SQL). In contrast, good analysts realise that they are really providing “services” and manage to meet the business needs just by keeping the time-value of information in mind. This, to the I-will-make-all-those-products-inhouse Director of Development, is equivalent of blasphemy.
If nothing else, this conversation has convinced me even more that the onus is on us to promote that collaborative and mutually-beneficial “IT and Analytics” approach. It’s a point very well made: “IT can happily exist without Analytics, but without IT there is nothing to do anayltics on.”
February 15th, 2006 at 10:08 pm
[…] This language barrier is particularly acute when business folks try to talk to IT folks. We’ve run into this problem a number of times. Here’s a good conversation on the topic. No solutions today, just venting… and laughing. […]