Author: Greg Baldini
The promise of BI
The promise of BI has always been better decisions through data. As BI professionals, we hope to empower business users to make new discoveries, ask better questions, and be able to answer those questions.
How many times do we see BI development end in a flop, though? Have you ever observed the battle between business users and IT? Why is it that the promise of BI always seems to cast a shadow over the delivery?
The waterfall methodology is the traditional approach to BI. Waterfall type development is made up of large, complete products in a single delivery. Also referred to as a “big bang” release, waterfall projects are typified by robust, enterprise-ready deliverables. That is the promise – with solid requirements and specifications, after development and testing, we will deliver complete and functional solutions.
Agile methodologies focus on end-user engagement, small, functional deliverables, and rapid iteration on those deliverables. Requirements and specifications are tackled in collaboration with end users, who provide continuous feedback throughout the short development cycles. This feedback is then incorporated into the next sprint. That is the promise – with continuous engagement, we will deliver functionality quickly and be responsive to feedback.
Mo’ methodologies, mo’ problems
Both methodologies have promise, and both have drawbacks in BI.
On the business side, users see IT and a waterfall approach as long, slow, and expensive, with an unknown payoff as a kicker. From IT’s perspective, business users have endless and constantly changing needs beyond development capacity. How long is your development backlog?
We face a tradeoff: an agile-only strategy addresses present business needs, whereas a waterfall-only strategy provides a robust enterprise architecture. Can we really do without either?
So how do we add agility?
Traditionally, the gap between these approaches has been filled by business analysts using Excel. Rob Collie observes that “Export to Excel” is the third most commonly used button in any BI tool, after “Okay” and “Cancel.” The analyst will take up the slack in BI development. Little reporting changes turn into huge tracts of business logic tied up in individuals’ Excel workbooks.
Eventually, we arrive in Excel hell. This is a situation I think we all know well.
The way out, perhaps unintuitively, to embrace its cause: business users can understand and discover their data and reporting needs better than IT.
Pick your poisons
We cannot solve the BI problem with either waterfall or agile methodologies alone. We can solve the BI problem with pieces from the waterfall and agile methodologies.
Modern BI tools empower end users to take control of data without relying on IT. Power BI, in particular, offers a powerful set of tools to facilitate collaboration between users and IT.
With Power Query and Power Pivot, an Excel power user can perform ETL and model data for immediate reporting needs. Business-driven development ensures constant feedback and facilitates the iterative approach that is proven so powerful in agile methodologies. Individuals and small teams meet their data needs in the short term with minimal IT involvement using modern self-service BI tools. Additionally, these tools offer a way for the business users to simultaneously work with IT.
The Power BI Service offers a collaboration portal – a sandbox that can be controlled and maintained by IT. It becomes easy to monitor usage and audit published reports.
When it comes time for enterprise development, there is an audit trail of ETL sources and processes. Power Query provides a list of each source and every transformation. Power Pivot enforces a tabular structure and defined measure logic, so can be seamlessly promoted into SQL Server Analysis Services. The final IT products can be exposed transparently in the same Power BI portal – a single pane of glass for all the organization’s data.
The business-driven solution easily feeds into IT’s development cycle, with clear, well-tested requirements and prototype implementations. This is how we create a hybrid agile approach, proven to be extremely powerful in increasing effectiveness of BI programs:
Companies that have complemented their traditional BI solutions with the agile approach are almost 50% more likely to consistently meet the needs of business managers than those that rely solely on traditional BI solutions.
Source: Agile BI: Complementing Traditional BI to Address the Shrinking Decision Window, November 2011, Aberdeen Group
The way we approach BI
Traditional IT waterfall delivery models do not completely translate to BI projects. However, in our experience, pure agile delivery models don’t work well for BI solutions either.
Just like IT projects are promoted dev -> test -> prod, BI solutions should be promoted individual / team -> department -> enterprise. With a hybrid agile approach, we can finally deliver on the promise of BI:
Better decisions through data.
If you would like to learn more about improving your business and empowering your end users, please feel free to contact us, or join us at one of our user group meetings. Microsoft BI User Group of Minnesota | Power BI User Group of Philadelphia