You say you want the real BI requirements?

We often preach how important effective and accurate business requirements are to the success if BI projects.  We also lament that gathering these requirements can be difficult.  There are any number of reasons for this. They range from analysts claiming “users can’t tell me what they want” to statements from customers like “just give me the data and I’ll know what to do with it.” That story usually does not end well.

Others like to play the Agile card and claim that documented requirements are not necessary and they will get to the right place through prototyping and iterative development.  That may happen in some cases, but it becomes hard to predict how long this will take and what the costs night be using this method exclusively.

Let me make a couple of suggestions that might help make your requirements more effective. The first has to do with process.  As business analysts, we are taught to discover, design, and document business processes so that we may find ways to enable them with information technology. But, you may say (and I have) that BI has nothing in common with a traditional business process like order-to-cash, for example. The result of the preceding question determines the next one, right? This lack of predictability makes traditional requirements gathering difficult if one goes about it using conventional methods.

Simply documenting capabilities is another approach, but it is a cop out.  How worthless to a developer is a spreadsheet that says “the System Shall…” about 150 times?

The fact is that we are in the business of supporting decision making and decision making IS a process. Think about the purchase decisions we all make.  We gather information from a variety of sources, some of it structured like price lists, supplemented by unstructured information like reviews and often aided by collaboration with friends over social media. This is a defined process that we enable with technology.

Business analysts should never begin requirements gathering by trying to mock up reports or defining the necessary data.  Instead, find out what the decisions are that are to be supported. Discover how they are made now, and then work to improve the process with BI technology.  Then the data acquisition, integration, presentation, and collaboration requirements will fall right out and you will end up with a document that takes a systemic approach to a better decision support capability.

I promised a second tip, so here it is.  When you set up requirements interviews with your customers, ask them to produce the spreadsheets they use now. Even if your customer has trouble telling you what they need, the spreadsheets usually speak volumes. They are the best evidence of how data should be managed and presented, as they are usually used to do what current reporting systems cannot.  Oh, and don’t be satisfied with just replicating what the spreadsheet is doing.  Think of them as prototypes that are limited by the technology that you are there to improve. 

Confusing BI with traditional IT applications development – 25 years later

Apologies for the delay between posts.  The long weekend afforded me the time to get back at it.

When I began my career as BI practitioner over 25 years ago, I came to it trained in Decision Sciences academically and from a professional background as a reformed financial analyst and IT business analyst / project manager.  This made me well qualified to take on early BI projects as, in the early days, most BI work was financial in nature. What I discovered pretty quickly is that traditional IT methods and skill sets did not translate very well to BI projects.  In fact, I had to unlearn much of what I had picked up managing transaction process application development in order to succeed with BI.

Part of this was the direct result of the technology.  ROLAP might have used the same DBMS technology as I was accustomed to, but data modeling took on a whole new meaning with de-normalized read-only star schema.  MOLAP?  That was another animal altogether.  Beyond that, the goal of an enterprise data warehouse required an eye toward not only the current project, but laying the groundwork for inclusion of additional subject areas regardless of how one went about that <insert Inman vs Kimball religious debate here>.

Far more important, however, was the need to ditch standard waterfall methodology in favor of iterative development cycles that would often start with a proof of concept, perhaps part of a vendor bake-off, followed by a prototype, followed by iterative builds of the application.  The good news was that the user interfaces came out of the box; some even had Excel sitting in front of them, so little development or customization was needed there.  This alone was a departure from what we were accustomed to.  All of the effort needed to be focused on the design and optimization of the underlying data structures, along with the extraction jobs from source applications that fed them.

This in turn created a need for an entire new kind of business analysis discipline that defined flexible environments to describe data navigation and investigation rather than pre-defined use cases of deterministic user interactions and transactions. Environments are defined by events, attributes and dimensions instead of business rules that characterize traditional requirements for transaction processing and simple reporting systems.  I elaborate on this in my previous post on Agile BI.

Project managers, for their part had to practice agility long before Agile became fashionable.  They learned quickly that their customers would not tolerate lengthy expensive data warehouse development cycles.  They demanded the rapid value the technology vendors promised.  This mandated breaking big projects up into smaller deliverables and iterative development that allowed users to experience capability they had never seen before and discover opportunities to create additional value even as they refined their original requirements.  Scope creep had to be assumed and managed, along with expectations.  Entirely new processes around data governance and end-user computing management needed to be developed.

BI developers came to understand that they needed to develop a certain tolerance for ambiguity in requirements and need for flexibility in their products, at times even predicting what the next set of requirements might be given their knowledge and experience with the underlying data.  This was a huge advantage on any BI project.

QA folks, for their part also needed to rethink their craft.  For one thing, it became necessary to work very closely with the BAs (if the BAs did not assume this responsibility altogether.) Assuring data quality is a very different task from testing transactions against a set of business rules. It puts an emphasis on automated comparisons with source data and working with users who have tribal experiential knowledge of the data as part of the user acceptance testing process.

So why bring all of this up now if it was all learned a generation ago? For one thing I have come to understand that there are still application development shops out there run by folks without direct BI experience that are just starting to take on BI responsibilities from their users. For another, recent technologies such as Hadoop have created a need to rethink aspects of the BI development discipline and the Agile movement has given the false impression that it translates literally to BI projects without modification.

I will comment in my next post on what characteristics to look for when staffing BI project teams in my next post. Until then, all comments welcome.

BI by any other name

I have a bit of an identity crisis with regard to what I do professionally. 

I have spent most of my career working within what we now call “Business Intelligence” or “BI” for short. Before that, this general discipline went by some other names, including Decision Support Systems (DSS) and Executive Information Systems (EIS). Since then the software marketing machinery, with the aid of the industry analyst community, has effectively retired those terms and coined a few new ones. These include Data Warehousing and Data Management for solutions that describe back-end platforms that support the user oriented front-end solutions like Analytics and Visualization. In general, though we tend to use BI to describe the process of acquiring, managing, processing and presenting information in direct support of decision making across business processes and organizational levels.

In recent years, I have focused on the Digital Analytics discipline and tools. This term is used to describe the art and science of capturing user interactions across platforms and touchpoints. We then integrate, aggregate, model and present this data to support decisions around things like marketing spend, personalization features, content management, and product experiments.

For me, the transition is relatively seamless as I think of Digital Analytics as simply a specialized form of BI. The basic processes are the same. We acquire the data, manage it, and present it in such a way as to discover patterns, model outcomes, and track results of previous decisions. The skills required are similar as well: You need Business Analysts who can bridge between the technical and business side personnel, developers who make the tools work, QA folks to verify the results and experienced managers who can keep everyone rowing in the same direction.

Since Digital Analytics is still a fairly young discipline, there is a perception out there that it is really something distinct and different from BI. There is some truth to this in the sense that the software tools universe is still pretty bleeding edge and fragmented - new big data/data science startups seem to appear on the scene almost daily - and few true data integration standards have emerged. This works to the advantage of relatively experienced practitioners and consultants who can command premium rates by promoting themselves as Digital Analytics experts.

We have seen this all before in the early days of recognizing BI as a discipline and a wave of OLAP and SQL-based reporting tools hit the market in the 1990’s. In those days, purchase and hiring decisions were often made outside of IT as Technology executives were slow to accept the importance of BI applications and the notion of end-user computing in general. Eventually, the tools market consolidated into a small number of dominant players and BI specialists became easier to find and identify. IT moved in to regain control of the spend and inject some needed discipline into application development and maintenance.

Currently, we see the investments in Digital Analytics driven mostly by business side organizations and facilitated by vendors who offer their solutions as a service that can be stood up almost entirely without the participation of IT. I think we will see history repeat itself. Business users are realizing that Digital Analytics data has limited value until it can be integrated with data from other business process like CRM, supply chain and ERP/Financials that IT manages. The best tools will continue to be absorbed into the enterprise software firms, and issues like data governance, privacy and security will need to be addressed by IT professionals who have the necessary experience. Beyond that, the market will force a balance between supply and demand for the specialized big data and predictive modeling skills that are scarce right now. Beyond that, those who are already skilled at application development, data presentation and interpretation in other domains will adapt their skills to the Digital Analytics space.

As the majority of businesses face their customers and partners using digital touchpoints, the information these interactions produce will become part of mainstream BI and digital analytics will become just another BI data domain. IT will embrace its role in managing that data, the technology will standardize, and the valued expertise will center on the data itself and the processes that add value to that data.

Self-service BI best practices

In my previous post, I discussed some of the drivers and enablers behind the current push within many IT organizations to implement or enhance “self-service” business intelligence and analytics. In addition to all the benefits for business users, IT also sees opportunities to benefit from driving down their development and maintenance burden, increasing speed to delivery, and freeing up resources to focus on infrastructure and governance responsibilities. Of course, this all depends on a successful implementation and roll out. This is far from a given, as many shops have discovered. There are, however, some best practices I can share that can enhance user acceptance and help generate demonstrable business value. I’ll break them out by people, process, and technology.

People: Adapting an IT organization to support a self-service BI model is often a major hurdle. It is generally accepted that some variation of a “hub and spoke” model is desirable. This implies a central group of shared data and application resources along with process experts. These combine with analysts directly aligned with one or more business units. It is not that simple though. I have seen several examples where a team of business analysts, project managers, and developers have attempted to stand up self-service delivery and found it impractical without a significant re-definition of roles. The key to success in self-service is to convert the organizational model from a development orientation to one focused on effective support and overall customer service. It is absolutely necessary to reserve enough bandwidth from all resources to remain engaged with users after applications are released or enhanced. Self-service environments are never quite finished. They must evolve as business requirements and priorities change and the size/composition of the user base changes. Initial successes will spawn new requirements as users see what is possible and new subject areas are folded in.

Ideally, the role distinctions between developers, business analysts, QA, and project managers break down in favor of a core group of BI practitioners who can perform in all of these roles to some extent. Ideally, the specializations shift to subject areas and business segments; e.g. Marketing, Finance, HR, CRM, Supply Chain, etc. This facilitates the alignment with user organizations. If practical, co-locating them is ideal. Cross training is also desirable to add necessary staffing flexibility. This type of support team can shift from the traditional reactive service model (“please fill out an enhancement request”) to a proactive one that is directly involved with business area decision processes and their users’ overall experience. They can resolve issues and train as necessary; while monitoring user acceptance, usage, and satisfaction. When additional development work or tuning is required, it can be handled directly by the dedicated spoke resources; working with the hub as necessary on shared tasks involving data acquisition and administration.

The other key consideration is to what extent business-side organizations can devote bandwidth to the support effort. Ideally, some can be carved out to support and train fellow users, help debug applications and data, and participate in data stewardship. In reality though, the roles do not change, only the organization that is providing the staffing and controls the budgets. Business organizations that are shifting from hosted to SAAS BI environments are likely better served by providing their own support staffing and relying on in-house IT hubs for data acquisition and governance. This tends to be a natural fit as the SAAS vendors tend to be more customer service oriented than in-house IT departments out of necessity.

Process: Customer-focused organizations enable customer-focused processes that begin within the spokes and work their way back to the hub. One example is training. If this solely takes the form of generic tools training for all users, it will not be as effective as training that is customized by role and business area. This can be easier than it sounds if a template is provided that includes all the basic concepts that can be customized and delivered by spoke staff to specific groups with similar tasks and requirements. Commonly desired enhancements to the templates, including those corresponding to application enhancements, are communicated back to the hub for inclusion in the templates.

An analogous approach can be applied to development of generic “accelerators”. These can include report and dashboard templates, portal designs, collaboration tools, data dictionaries, provisioning models and security schema. One practice I highly recommend is a “promotion” process for successful designs. For example, when a report or dashboard design gains high acceptance within one business area, it can be adapted to others or become an enterprise standard that is centrally maintained.

This brings up another important success factor for self-service BI: Creation of a formal group that includes representation from the hub, the spokes, and accomplished users that collaborates actively and meets regularly either physically or virtually on a regular basis. I like to call it a “community of practice” but I have seen it go by many other names, including “center of excellence” although that one always sounded a little pretentious to me. These groups are very effective for knowledge sharing, promotion of the technology tools, and overall two-way communication with the user community at large. I also encourage members to show successful new applications of the technology, tools, and models off to the group. This is a great way for architects within the hub to stay close to trends and changes in business requirements.

One note about the testing process: It also has centralized and de-centralized aspects, but it is a bit different in the sense that integrated systems tests usually involve the entire user community and requires a high level of coordination and central project management. As such, it generally requires central administration with user area validation.

Technology: I do not have as much to say about technology because, contrary to popular misconception, the technology choices and system architecture are generally not a key to success. In fact, on occasion I see technology platforms completely replaced to solve what is really a process issue. Most, if not all of the mainstream tools and data platforms out there can underpin a successful self-service platform, as long as there is a well thought out user interface that is easy to use. Often, this is in the form of a portal; but can be as simple as the universal front end known as Excel. The goal is remove barriers to availability and utility by making:

1. your applications easy to acquire and use on multiple devices
2. your data reliable, transparent, relevant, and timely, and
3. help available when and where it is needed

My last post in this series will discuss some of the favorable and unfavorable environmental factors as it relates to standing up a self-service BI environment.

Some thoughts on self-service BI

Let me begin this series of posts with some historical context:

Ever since the earliest days of what we refer to as self-service computing, there has been controversy in Corporate America around the relative roles of IT and the user (Business) departments in the development and administration of applications that support knowledge workers. This balance of power has been disrupted by several waves of technology, but the basic issues are always the same and persist today. In the beginning, there was pure mainframe computing; not really accessible to end users unless they were willing to write 3^rd generation language (e.g. COBOL) programs on punch cards, feed them into readers, and then wait for a job to complete and get results out of a printer. Then came time sharing - the original clouds - which made things a bit more accessible and immediate, but IT was still in firm control. Even then, some business departments like Finance started hiring tech-oriented staff to configure and administrate highly customizable applications like general ledgers and the early ERP systems. These also featured proprietary programming languages that completely blurred the line between configuration and full-on application development.

Self-service computing became a wave, and quickly gained the most traction within decision support applications that were renamed ‘Business Intelligence’. IT, fearful that ‘amateurs’ were putting critical processes at risk, moved to take over that activity. This over the objections of business side management that valued their control over these functions and priorities. Turf wars ensued.

Things only got more complicated as higher level “4^th generation” data handling, planning, and modeling languages, like SAS and FOCUS, were marketed to end-user programmers and their management that paid for them. At this point, there was no turning back on the expectation that application development could occur outside IT. One common reaction was for IT to create entirely separate environments, based on somewhat smaller and cheaper hardware that used extract data from the mainframes to perform dedicated modeling, planning and reporting activity in real time. This enabled dedicated end-user computing organizations that needed IT support and feeding; but otherwise operated independently as long as they stuck to supporting non mission-critical applications. (In practice, more than a few slipped through.) One immediate consequence was a massive proliferation of often redundant data extracts. IT responded by building data warehouses in the hope of regaining some order and control over the source data, if not what happened downstream.

From there, the PC revolution came along with spreadsheets, databases and more powerful higher level languages. Another layer of individual autonomy was created with all the attendant risks and opportunities. Over time, the PCs became smaller and more interconnected through wired and wireless networks until they became completely mobile, not to mention owned by users. Things became really chaotic and limited at the same time in the sense that these devices cannot natively support the level of collaboration we want or capacity to process the huge volume of transaction data we now collect.

IT was and is still saddled with the responsibility of maintaining data integrity, security, and availability across a set of processes operating over devices, applications, and networks largely outside of their direct control. Currently, this trend is being extended to main operational applications, from which internal data is still primarily sourced, as they are relocated to SAAS, or software as a service clouds. In a sense, the technology has come full circle, but the basic dichotomy of how to balance control, responsibility, resources, and workload between IT and users remains.

The other major factor driving demand for true end-user BI is the service that we have come to expect as consumers via the Internet from the likes of Google that understand natural language and make an enormous variety and volume of the world’s knowledge and data searchable and available on seconds notice. We then use social networks to share the knowledge and collaborate with friends and colleagues. We want that same power and ease of use in the workplace. As BI professionals, we strive to provide such capability that we refer to these days as self-service BI; and the software cloud service vendors are trying to sell it to us.

In the next post, I will detail some of the best practices that I have learned from organizations that have put successful self-service BI capabilities in place across many diverse technology architectures.

12 traits of really effective BI Portals

As BI professionals, I think we have a tendency to devote our time, effort, and attention to building our applications from the inside out. We really like to focus on creating really great architectures and engineering applications that emphasize performance, capacity, flexibility, reliability and accuracy. This is how we, and perhaps our peers and superiors, tend to evaluate us and it is certainly what our vendors like to promise us. It is not, however, what our customers care about as it is not what they see or directly experience. For them, now that virtually all BI applications are web and/or mobile enabled, the application user interface (UI) is the experience. An otherwise good application with a bad UI will not create much in the way of customer satisfaction.

It is tempting to cop out a bit on this by relying on your BI application vendor to develop the UI and simply go with it. There are a couple of problems with this. One is that the leading vendors in the BI space have not really prioritized their UI, and it shows. Most provide the capability to customize the UI or embed it within another application, but this is not done often enough. The other problem is that our users are rarely confined to a single tool despite all attempts to standardize. As a result, they must adapt to multiple inconsistent UIs and this turns them off.

One proven way to create a delightful UI that effectively supports the process of decision making is to embed your applications within a single coherent well-executed portal. There are many ways to go about this, but the successful ones have many traits in common.

1.       They are highly customizable, but with common content and look and feel

BI portals on the web are just like any other site in that they are more effective when some level of personalization is possible. One size does not fit all. Different jobs and roles have different requirements, and everyone likes to tweak those looks to their taste. Some functions and data will be common to all though. The look, feel and reference data should be consistent across all roles to ease transitions and maintain intuitive navigation (See #3.)

2.       They are easy to use. Little or no documentation or training required

Not much to explain here. If your portal requires extensive documentation or training, it is not intuitive enough. If the functionality is so complex or sophisticated that specialized training is needed, you have developed an analytic tool, not a general BI portal.

3.       They are easy to navigate with search that really works

It is hard to overstate the importance of this. We can Google so much of the world’s knowledge, why not our company data? Failure here will overwhelm your support operation.

4.       They have help that helps; and someone to call when it doesn't

Don’t rely on vendor help content. Customize it and make it relevant to your users and roles. Include FAQs and update constantly.

5.       They are trustworthy, relevant and well governed

This is another one that is not optional. Nothing kills any BI application faster than a reputation for having bad information. Make sure your sources are documented so users know whom to call with questions. Obsolete data is almost as bad. Assign responsibility for removing what is no longer valid, needed or just plain noise. Don’t let this fall victim to budget cutting.

6.       They are fast

Speed is taken for granted these days on the web. That does not mean all queries are sub-second, but set expectations properly – people also understand cost/benefit when it is communicated and negotiable.

7.       They are responsive to multiple devices

Like everyone else these days, we must accept the reality that access must be supported on multiple screen types. That does not mean we just shrink the experience for phones though. Use cases vary by device.

8.       They handle structured and unstructured data equally well

It’s not just about tables and pie charts anymore. We can now process and manipulate images, video, text and audio. All of these can support better decisions and should be available to query.

9.       They do not try to be all things to all people

Trying to do too much creates complexity. If a requirement for one group or role falls too far outside your architecture, develop it outside the portal with a link. This is particularly true of advanced analytics, data discovery and modeling functionality.

10.   They are secure

The paradox here is that we work hard to create applications that make key information easily accessible to all who need it. If we are not careful, we create one-stop shopping for those who might steal or abuse it. It does not have to be a usability burden if done right, but fight any temptation to under-invest here.

11.   They promote collaboration

Decision making and modeling are inherently collaborative processes. How many endless email threads must we wade through to get the benefit of group input and multiple viewpoints? Our BI vendors have only recently started to recognize this, but portal software is usually the best tool for this job.

12.   They are monitored, supported and evolve over time

No matter how well you execute, your experience can always get better. Get feedback and listen to it. Have your support team reach out on a regular basis proactively to see what is working out there and what isn't. Establish a testing program for usability enhancements as well as new functionality. Take a lesson from commercial sites and devote some development bandwidth to small improvements that can be done outside the project schedule and budget. This way you can develop experiments and deploy successful ones in production.

These may seem like worthy, but pretty lofty ambitions that are very hard to achieve. It is far from impossible. There are many examples of successful internal corporate information portals that have most or all of these traits; but if you have not worked at one of those firms, you may not have seen one. If you need an example that we can all see, consider the portals that the consumer investment services firms provide for their customers. Sites like Fidelity, Schwab, and their peers have been in the business of supporting the investment decisions of their customers online for many years and have devoted considerable time and effort to optimizing their experiences to the point where they do all of these things well. If you are a customer of one of these firms, consider their sites and apps in relation to these 12 traits and I think you will see what I mean.

As always, all comments and discussion welcome.

Do Agile methodologies and Analytics really mix?

There have been many good articles, papers and posts on this subject - see here and here for examples.  Beyond that, there is a whole sub-methodology out there known as Agile Analytics that has been formalized. Experienced practitioners are just about unanimous in the opinion that a certain amount of agility (small a) is mandatory for data warehousing (DW) and Business Intelligence/analytics (BI/A) projects.  For my part, I have always rejected the idea that traditional waterfall methodology applies effectively to BI/A projects and have resisted all attempts to impose it on my programs long before anyone was talking about Agile. Some of the major reasons include:

• BI requirements in general, and analytics requirements in particular, are generally somewhat fuzzy at the outset; users often cannot be specific about what they really want until they actually see what is possible in the form of a functional application
• Formal requirements definition, can often skipped in favor of rapid iterative prototyping - I should also be quick to point out that this does not remove the need for documentation; more on this later
• This, in turn really mandates the kind of iterative, rapid development cycles and ‘chunking’ of projects that are pillars of the Agile method
• Turning your program into a portfolio of shorter-term deliverables promotes early and continuous value delivery and ‘envy’ marketing internally
• Small teams with members who can play multiple roles (design, code, test, etc.) that include both technical and business (user) expertise are also very well suited to analytics development
• Knowledge sharing an mentoring among the teams is also uniquely advantageous for analytics, in part because of the need to continuously evolve analytic applications after release, and sustain supporting processes like data governance
• This model also discourages fine-grained role definition that often relegates junior staff to occasionally mind-numbing roles like QA and report/dashboard development
• Most importantly; I have seen too many BI/DW projects die of their own weight when they broke the rule that says you need to deliver something of value to your users within six months or risk losing your sponsorship, funding, or even relevance

So, end of story… Agile works for analytics, and all analytics shops should welcome an Agile development mandate from on high right?  Not so fast…  In my experience, Agile (capital A) is not perfectly suited to BI/A in a number of ways, mostly because it was not really developed with BI/A projects in mind.

Unlike more conventional software projects where Agile has been so successful, in BI/A programs we often build environments as opposed to applications. We create the ability for others, often users, to build the actual applications in self-service mode.  Environments are specified as events/elements, dimensions and hierarchies; along with initial outputs like reports and dashboards. Please note that I am generalizing here; there are notable exceptions to this such as embedded operational BI applications within process automation. An example would be fraud scoring within customer service applications.

Then there is the fundamental Agile construct that is the user story. This idea needs to be redefined in a BI/A context.  We often do not know in advance exactly how our environments will be used – as the answers usually beget the next questions when engaging in data discovery and data investigation.

Another BI/A trend that works against Agile is toward offshore development.  Generally, offshoring works best when requirements can be locked down and documented with great rigor.  Agile works best when all stakeholders can be co-located despite recent advances in collaboration technology.

Another factor that is often overlooked is the Agile mandate that projects are strictly time-boxed and code that is at least testable be delivered on short cycles, typically 2-3 weeks. This is often not entirely practical in the BI/DW context. Large projects can, and should be chunked up into smaller deliverable cycles; we generally accomplish this by dividing up the work by data subject area or decision class. If necessary, we can split development between a data infrastructure track (DW/ETL development) and a parallel track to develop the user facing pieces.  Prototyping can be done using sample data extracts until the underlying data services infrastructure is complete. This all helps, but strict timeboxing can be hard to enforce, particularly for the infrastructure work, because of the dependencies on data source applications, and their schedules, which must be accommodated.

Another difference is that BI/A projects are often evergreen in the sense that they in a constant state of enhancement to adapt to changes in source data and decision scenarios; that is, the line between development and maintenance becomes blurry.  It helps immensely to keep the core development team intact for this and not send them off to the next new Agile project.

You may be thinking that this all sounds pretty consistent with Agile philosophy, you just need to modify the orthodoxy a bit to fit in the BI/A context.  I agree with that.  In fact, I really thought Agile would be a great benefit to BI/DW practice since it discouraged waterfall thinking, but it became a case of ‘be careful what you wish for’, at least as it relates to the websites and mobile applications that are my main data sources these days.

When the applications (and websites) we are looking to instrument and analyze are developed using Agile, far more often than not one of the requirements that never gets properly documented or missed entirely is the one that specifies the need the capture the necessary event data properly.  Why?

• This requirement does not fit very well into the Agile user story paradigm; the user of the source application is usually not the user of the generated data
• Scrum masters are rarely trained to recognize the need to generate data as a fundamental requirement of the application or product. If they do, it gets assigned a low priority especially when schedules get tight
• Data requirements can span the entirety of the application or website; as such the requirement applies to multiple scrum teams working mostly independently of each other
• Testing becomes especially challenging since the modules of the application must be tested along with the downstream analytic application in an integrated fashion once all the pieces are built
• It becomes necessary to identify the analytics team as a ‘user’ that gets to participate, test and iterate just like any other business user. This works best if the analytics team is part of the scrum, but most analytics teams are spread too thin to make that practical

So how do we reconcile this?  Agile is here to stay.  If there is an easy way, or one that has worked for you, please let me know.  In the meantime, all I can suggest is that you try to work with Program and Scrum Masters that understand the notion that their applications are not fully successful if they do not yield the data necessary to analyze customer/user behavior and overall process effectiveness.  Best case, the development and analytics programs become integrated to the point where the analyst looking to perform data discovery or generate performance metrics becomes a fully supported user story, with all that implies upstream.

  

  

Why do analytic application requirements have to be so hard?

In my last post, I blamed the failure of many analytics and BI software implementations on a failure of requirements definition.  In this segment, I will discuss why I think this happens as often as it does and ways to avoid this costly result.

There are many common misconceptions out there in the software development community around analytic applications.  One of the worst is that proper requirements definition is not all that important and that you can approach it as a “lite” version of a traditional requirements definition process that is used for more process-driven projects.

Another one has to do with the basic purpose of requirements definition.  It is often treated as just a perfunctory step mandated by a pre-defined methodology that only a time consuming set of interviews and a long deliverable document can satisfy.  Worse yet, this might consist of an endless list of line items beginning with “the system shall…” I find these to be nearly useless in the context of analytic applications.  Defenders of this approach will counter that these are only “Functional Requirements” and they are simply a prelude to a set of “Technical Requirements” that actually become the inputs to the development phase. They go on to point out that this step helps the users get a handle on, as well as prioritize their own needs and desires.  That’s acceptable if the technical requirements are actually developed, but how often does that happen with any real rigor?

This cookie-cutter approach has never worked for me, whether we are talking about generic BI applications or specific analytic applications like digital analytics.  It also makes little difference whether we are talking about implementing a pre-packaged application or building one from scratch.  What I try to stress on the projects I am responsible for is that requirements, no matter how they are documented and communicated, exist for the sole purpose of guiding those who will write the code and/or configure the rules for your application.  Failure here equals project failure. Period. Bad requirements lead to test fails, bad data, delays, cost overruns and unhappy users.  This not only means you have to get the actual requirements right, but doing so at the right time and in a form that is most useful to your developer(s).  This requires getting their input on this and listening to it.

So where is the disconnect here? I think it is the tendency of those project managers and business analysts with most of their experience working operational software projects, to apply the same techniques to an analytic application.  This is like trying to drive nails with screwdrivers.  Operational applications like general ledgers and reservation systems are generally procedural or even transactional in nature with well-defined and static processes and user roles.  One step follows the next in a predictable way and can be depicted with flow charts and swim lanes.  In BI, the ‘use cases’ and roles are dynamic, even unpredictable in nature. The analytic process tends to be iterative (the answer to the last question defines the next question), collaborative, and personal in the sense that different users like to present the same information in different ways.  Even so-called ‘operational’ BI, where the decisions to be supported are pre-defined, has to tolerate edge case situations where the decision trees sprout new branches on the fly.

One alternative approach is to fall back on our instincts and begin the requirements definition process by trying to envision what the user should actually see, and working backwards from there.  That works pretty well if we are talking about a reporting application (not to be confused with analytics – a distinction for a future post) where the desired displays can be mocked up and become the de-facto requirements for the data modelers and report writers to work from.  Of course, this only works if the reports never change materially. The point is we are not specifying static processes or reports.  We are specifying environments that must be flexible, dynamic, and modifiable by people other than those who developed them.  These environments are defined by:

•          The events to be captured,
•          The attributes of those events we need, and
•          Any necessary context – this will vary by application type, but using customer analytics examples:

-    The event - when they occurred and in what sequence?

-    The customer/user contact  - where did it come from?

-    The customer/user origin – what role, segment or campaign?

-    The property or platform  - what device or what  brand?

•          Derived Metrics, like variances, rates, or indices
•          Any required projection or prediction algorithms
•       (If applicable) Connection points to operational systems for quick action on delivered insights (e.g. media purchases)
•          Other downstream application requirements your system is feeding, these could be:

-    Personalization tools

-    Pricing tools

-    Testing tools

-    3^rd party  website tags

•          So called “non-functional” (I love that term) requirements like security and retention periods
•          For application configuration, we can add:
•          Presentation preferences (tables, graphs)
•          Menu structures
•          Historical comparisons

That’s pretty much all there is to it.  If you get these things right, you will have all the necessary building blocks in place in your application to construct a satisfying experience for your stakeholders at the outset.  It may not be perfect, but the fixes and enhancements identified in testing or post launch should be relatively easy to implement.  On the other hand, if you miss on the basic data acquisition and manipulation requirements, you will likely end up ripping up quite a bit of track.

A note for those of you who design and implement these applications as part of a business organization, that is, outside of IT.  Don’t fall into the trap of thinking just because you are not subject to the tyranny of a formal and rigorous methodology for applications development means you should operate without any process at all. At minimum, you need:

•          A project plan
•          Defined project roles and responsibilities
•          A template for your documentation that all project members are comfortable with
•          A complete test plan that uses the functional and technical requirements as a basis and includes an integrated testing cycle that uses the event data, the processing rules and the presentation layer after the code for all those elements is frozen

Winging it with folks who have never done this before almost never ends well.  Analytic applications that deliver really valuable insights are deceptively complex and represent a significant investment. They require a defined implementation process using experienced folks.  If you do not have them, hire some.