Thursday, October 12, 2017

Why are Technology Managers walking away from Analytics?

As we all know, a robust business Intelligence and analytics capability has become a major priority for global enterprises. Yet, in a very real sense, Technology managers (IT) are actively distancing themselves from it. How do we explain this paradox? Is it a case of liberation or abdication?

The two megatrends in analytics are self-service, where IT pushes application development out to users and Cloud-based analytic platforms, where IT outsources hosting to a service provider or tool vendor. Both of these dramatically reduce the role of IT in analytic application development, delivery and maintenance.

Why is this happening? There are several factors at work. One is something we call Data Democratization. This is just another way of saying that the massive amounts of data business now produces and highly values can become directly accessible to business consumers and freed from IT ‘tyranny’. IT customers love this idea.  Business units gain the control they seek over their own budget and priorities. Let’s call this Do-It-Yourself (DIY) analytics.

Recent developments in technology enable DIY analytics. The custom hardware and specialized skills once needed to acquire and warehouse data are being replaced by wave of specialized products that can:

  • Acquire data using pre-defined connection software. No technical expertise required.
  • Store, process, and analyze very large volumes of structured and unstructured data using low-cost commodity hardware
  • Retrieve data, visualize results and build applications efficiently and quickly without the need for optimized data models and custom coding. Professional developers no longer needed.

Technology is not the only driver here. Organizational factors also make this model appealing for IT. When IT ‘liberates’ analytics, it has: 

  • Less budget to justify
  • One less source of backlogs and maintenance headaches
  • More time to focus on other issues like security and disaster recovery
  • Created a sense of agility within the enterprise with more responsive application development

The market for analytics technology has also become biased toward users and away from IT. Although the tools have become much easier to use, they have also become more difficult to select and administer as they become more diverse and complex. They are expected to support everything from data warehousing to reporting, advanced visualization, collaboration, predictive modeling, and much more. Matching these capabilities with business requirements requires in depth knowledge of business processes. Beyond that, the tools market has become more fragmented, with many startups and specialized, industry-specific products.  Vendors, for their part, prefer selling to users in general as these sales cycles tend to be faster.

Are these trends a good thing for your organization? It depends on both the situation and the quality of execution of a decentralized DIY BI strategy, but there are some important things to consider in every case. Governance is the elephant in the room. Striking the right balance between IT and user control is key. Risk management and overall cost controls are best managed centrally. Moreover, sound data management dictates that raw data are enterprise level assets for all to use, and thus should be managed centrally, or at least through some kind of federated model to assure quality and integrity.

Another consideration is accountability. If a self-service analytics model fails, IT will likely get the blame. As such, IT needs to do all it can to assure success and a smooth and responsible transfer of responsibilities.

Here are some recommendations: Analytics is not a technology or an application. It is a capability with people, process, and technology components. Organizations within an enterprise become partners to create this capability by doing what each does best. For example, IT is usually best positioned to maintain and manage data quality, security standards, tools training, systems administration, and vendor management. The business units can then apply the technology to support their decisions and business processes most effectively. These organizations also need to cooperate and, to some extent, govern themselves with regard to things like: knowledge sharing, vendor relationships, data sharing and certification of results.

There is one other thing to keep in mind: The shifting of responsibilities for analytics is not a new phenomenon. Control of analytic applications has been something of a cyclical tug of war between users and IT since the early days of timesharing, the PC and spreadsheets. At this point, we are near the peak of user control in the cycle. History tells us this may reverse in time if governance fails and breakdowns in security and trust force a return to central control.

x

Tuesday, April 18, 2017

Data Scientists vs BI Analysts: Why is this a thing?

I came upon this popular Linked In post that attempts to define and draw distinctions between these two roles. It is one of several I have read recently.

Upon reading this, my initial reaction was how much I disagreed with these characterizations. My feeling is that, if we want to draw such distinctions at all, they should be between business analysts and data analysts. Data Scientist is just a newer title that combines some attributes of both business and data analysis. It nearly always includes a mastery of big data technologies and statistical methods, thus commanding higher compensation.

Then I realized that this is all beside the point. These role definitions are more about recruiting, HR job descriptions, org charts, and pay grades than what is actually required to succeed in an analytics program. What matters is having the necessary skill sets on the analytics team, regardless of what roles or organizations they come from.

As has always been the case in BI & Analytics, the critical skill sets can be considered using the classic Input à Process à Output model:

Input
Process
Output
·   Data sourcing & extraction (ETL/ELT)
·   Data preparation
·   Data quality
·   Data governance
·   Data navigation & investigation
·   Data discovery
·   Business analysis
·   Modeling
·   Predictive analytics
·   Reporting
·   Dashboards & KPIs
·   Visualization
·   Operational applications
·   Presentation/storytelling

BI/Analytics technology no longer respects the walls between these skill sets. The market has moved away from niche tools to suites that address the entire analytics capability set. For example, what were once pure visualization tools now offer data sourcing, transformation and modeling features. The impact of this has been to democratize the entire data supply chain in such a way that it has moved much closer to the business and completely obscured the role distinctions between data analysts, scientists and yes, decision makers. In fact, the overlap of these roles and the trend toward self-service BI tends to create organizational redundancy within larger organizations that can afford it.

The fact that the technology is available to many roles does not mean that individuals should be expected to have all the necessary skills to leverage it effectively. In fact, very few people do. Our trade has always placed a high value on those who can navigate data, develop actionable information, and present it effectively because they are still rare. This won’t last. The generation that is now entering the workforce has a much higher level of data skills than its predecessors and will value the ability to develop their own stories and support its own decisions with data as it rises to executive positions.

If the goal is to leverage data most effectively and maximize decision support success, don’t look to your organization to create a new role. Look to your team to fill any skills gaps, preferably by expanding the roles already in place. The goal is to minimize the organizational distance, handoffs, and filters between your sources of data and those who directly put it to use in business processes.