Sunday, June 29, 2014

BI by any other name

I have a bit of an identity crisis with regard to what I do professionally. 

I have spent most of my career working within what we now call “Business Intelligence” or “BI” for short. Before that, this general discipline went by some other names, including Decision Support Systems (DSS) and Executive Information Systems (EIS). Since then the software marketing machinery, with the aid of the industry analyst community, has effectively retired those terms and coined a few new ones. These include Data Warehousing and Data Management for solutions that describe back-end platforms that support the user oriented front-end solutions like Analytics and Visualization. In general, though we tend to use BI to describe the process of acquiring, managing, processing and presenting information in direct support of decision making across business processes and organizational levels.

In recent years, I have focused on the Digital Analytics discipline and tools. This term is used to describe the art and science of capturing user interactions across platforms and touchpoints. We then integrate, aggregate, model and present this data to support decisions around things like marketing spend, personalization features, content management, and product experiments.

For me, the transition is relatively seamless as I think of Digital Analytics as simply a specialized form of BI. The basic processes are the same. We acquire the data, manage it, and present it in such a way as to discover patterns, model outcomes, and track results of previous decisions. The skills required are similar as well: You need Business Analysts who can bridge between the technical and business side personnel, developers who make the tools work, QA folks to verify the results and experienced managers who can keep everyone rowing in the same direction.

Since Digital Analytics is still a fairly young discipline, there is a perception out there that it is really something distinct and different from BI. There is some truth to this in the sense that the software tools universe is still pretty bleeding edge and fragmented - new big data/data science startups seem to appear on the scene almost daily - and few true data integration standards have emerged. This works to the advantage of relatively experienced practitioners and consultants who can command premium rates by promoting themselves as Digital Analytics experts.

We have seen this all before in the early days of recognizing BI as a discipline and a wave of OLAP and SQL-based reporting tools hit the market in the 1990’s. In those days, purchase and hiring decisions were often made outside of IT as Technology executives were slow to accept the importance of BI applications and the notion of end-user computing in general. Eventually, the tools market consolidated into a small number of dominant players and BI specialists became easier to find and identify. IT moved in to regain control of the spend and inject some needed discipline into application development and maintenance.

Currently, we see the investments in Digital Analytics driven mostly by business side organizations and facilitated by vendors who offer their solutions as a service that can be stood up almost entirely without the participation of IT. I think we will see history repeat itself. Business users are realizing that Digital Analytics data has limited value until it can be integrated with data from other business process like CRM, supply chain and ERP/Financials that IT manages. The best tools will continue to be absorbed into the enterprise software firms, and issues like data governance, privacy and security will need to be addressed by IT professionals who have the necessary experience. Beyond that, the market will force a balance between supply and demand for the specialized big data and predictive modeling skills that are scarce right now. Beyond that, those who are already skilled at application development, data presentation and interpretation in other domains will adapt their skills to the Digital Analytics space.

As the majority of businesses face their customers and partners using digital touchpoints, the information these interactions produce will become part of mainstream BI and digital analytics will become just another BI data domain. IT will embrace its role in managing that data, the technology will standardize, and the valued expertise will center on the data itself and the processes that add value to that data.

Monday, June 9, 2014

Self-service BI best practices

In my previous post, I discussed some of the drivers and enablers behind the current push within many IT organizations to implement or enhance “self-service” business intelligence and analytics. In addition to all the benefits for business users, IT also sees opportunities to benefit from driving down their development and maintenance burden, increasing speed to delivery, and freeing up resources to focus on infrastructure and governance responsibilities. Of course, this all depends on a successful implementation and roll out. This is far from a given, as many shops have discovered. There are, however, some best practices I can share that can enhance user acceptance and help generate demonstrable business value. I’ll break them out by people, process, and technology.

People: Adapting an IT organization to support a self-service BI model is often a major hurdle. It is generally accepted that some variation of a “hub and spoke” model is desirable. This implies a central group of shared data and application resources along with process experts. These combine with analysts directly aligned with one or more business units. It is not that simple though. I have seen several examples where a team of business analysts, project managers, and developers have attempted to stand up self-service delivery and found it impractical without a significant re-definition of roles. The key to success in self-service is to convert the organizational model from a development orientation to one focused on effective support and overall customer service. It is absolutely necessary to reserve enough bandwidth from all resources to remain engaged with users after applications are released or enhanced. Self-service environments are never quite finished. They must evolve as business requirements and priorities change and the size/composition of the user base changes. Initial successes will spawn new requirements as users see what is possible and new subject areas are folded in.

Ideally, the role distinctions between developers, business analysts, QA, and project managers break down in favor of a core group of BI practitioners who can perform in all of these roles to some extent. Ideally, the specializations shift to subject areas and business segments; e.g. Marketing, Finance, HR, CRM, Supply Chain, etc. This facilitates the alignment with user organizations. If practical, co-locating them is ideal. Cross training is also desirable to add necessary staffing flexibility. This type of support team can shift from the traditional reactive service model (“please fill out an enhancement request”) to a proactive one that is directly involved with business area decision processes and their users’ overall experience. They can resolve issues and train as necessary; while monitoring user acceptance, usage, and satisfaction. When additional development work or tuning is required, it can be handled directly by the dedicated spoke resources; working with the hub as necessary on shared tasks involving data acquisition and administration.

The other key consideration is to what extent business-side organizations can devote bandwidth to the support effort. Ideally, some can be carved out to support and train fellow users, help debug applications and data, and participate in data stewardship. In reality though, the roles do not change, only the organization that is providing the staffing and controls the budgets. Business organizations that are shifting from hosted to SAAS BI environments are likely better served by providing their own support staffing and relying on in-house IT hubs for data acquisition and governance. This tends to be a natural fit as the SAAS vendors tend to be more customer service oriented than in-house IT departments out of necessity.

Process: Customer-focused organizations enable customer-focused processes that begin within the spokes and work their way back to the hub. One example is training. If this solely takes the form of generic tools training for all users, it will not be as effective as training that is customized by role and business area. This can be easier than it sounds if a template is provided that includes all the basic concepts that can be customized and delivered by spoke staff to specific groups with similar tasks and requirements. Commonly desired enhancements to the templates, including those corresponding to application enhancements, are communicated back to the hub for inclusion in the templates.

An analogous approach can be applied to development of generic “accelerators”. These can include report and dashboard templates, portal designs, collaboration tools, data dictionaries, provisioning models and security schema. One practice I highly recommend is a “promotion” process for successful designs. For example, when a report or dashboard design gains high acceptance within one business area, it can be adapted to others or become an enterprise standard that is centrally maintained.

This brings up another important success factor for self-service BI: Creation of a formal group that includes representation from the hub, the spokes, and accomplished users that collaborates actively and meets regularly either physically or virtually on a regular basis. I like to call it a “community of practice” but I have seen it go by many other names, including “center of excellence” although that one always sounded a little pretentious to me. These groups are very effective for knowledge sharing, promotion of the technology tools, and overall two-way communication with the user community at large. I also encourage members to show successful new applications of the technology, tools, and models off to the group. This is a great way for architects within the hub to stay close to trends and changes in business requirements.

One note about the testing process: It also has centralized and de-centralized aspects, but it is a bit different in the sense that integrated systems tests usually involve the entire user community and requires a high level of coordination and central project management. As such, it generally requires central administration with user area validation.

Technology: I do not have as much to say about technology because, contrary to popular misconception, the technology choices and system architecture are generally not a key to success. In fact, on occasion I see technology platforms completely replaced to solve what is really a process issue. Most, if not all of the mainstream tools and data platforms out there can underpin a successful self-service platform, as long as there is a well thought out user interface that is easy to use. Often, this is in the form of a portal; but can be as simple as the universal front end known as Excel. The goal is remove barriers to availability and utility by making:
  1. your applications easy to acquire and use on multiple devices
  2. your data reliable, transparent, relevant, and timely, and
  3. help available when and where it is needed

My last post in this series will discuss some of the favorable and unfavorable environmental factors as it relates to standing up a self-service BI environment.