Business Intelligence (BI) projects are usually intended to meet a hybrid of different needs, frequently including standardized reports and dashboards, self-service reporting, and data discovery tools. Quite often, unexpected relationships and architectural challenges are found in data during the course of a project. Experienced BI teams are not strangers to mid-project discoveries such as many-to-many relationships instead of traditional dimensions, key relationships that do not map properly, and calculations that need to be modified for the unique needs of the business users.
Despite detailed working sessions with business users and preliminary data analysis, sometimes changes to requirements are needed for a project to meet the needs of the business. Even the most carefully planned projects can end up with initial SOWs that don’t fully capture the details of the actual development required to complete a project.
That’s why there is a process for change requests, right? While a change request process is an essential part of any BI project, the limitations of a project budget can eventually impact the ability for a project to succeed. Changes requested to reports and dashboards during the testing phase of a project will often lead to the discovery of ETL and cube modifications that are required in the underlying architecture of the entire project.
So how do you avoid a situation in which an endless cycle of development re-work iterations creep up upon a diminishing budget? Careful and strategic management of deliverable expectations are essential to successful BI projects.
Key deliverables for BI projects often include dashboards, reports, and solutions such as cubes and data warehouses. The following graph shows Earned Value (EV) as a percentage of initial expectations vs Time for an individual BI project deliverable. Notice that during a project timeline, a deliverable will often increase in EV slowly as the project begins, then increase during development, and then slow down during testing and handover:
Let’s consider three hypothetical high-level deliverables: 1) Quality Metrics, 2) Cost Metrics, and 3) Customer Satisfaction Metrics and put them on this chart with EV as a percentage of initial expectations on the y axis:
Notice that the earned value of 1) Quality Metrics has exceeded expectations, 2) Cost Metrics has met expectations, and 3) Customer Satisfaction is below expectations. Let’s say the estimates for typical Earned Value (EV) as a percentage of initial expectations at the end of development are as follows:
If two of the three deliverables meet expectations, should a project be considered a success? It is a question that cannot always be directly answered. Here are some examples of the types of feedback that business users might have about these deliverables:
Deliverable #3 can lead to some difficult choices. A project team will either need to 1) submit a change request affecting the architecture of the entire solution, or 2) accept the shortcomings of Deliverable #3 for the current project.
There are several risks which must be considered when making late-project changes to a BI solution that impact the underlying architecture:
- Deliverables such as ETL, EDW, and OLAP/Tabular solutions might already be approved, and will now require rework.
- The risk of breaking or impacting other parts of the solution must be assessed.
- Costs will usually go up.
- Project duration will likely be extended. Adding more resources to a project usually does not impact schedule in such a situation.
- The new architecture might not be perfect, which could lead to a seemingly infinite loop of change requests and re-work.
Situations such as this are not uncommon in BI. BI, by nature, will uncover previously unknown relationships and anomalies in data. This situation needs to be managed carefully from a project management perspective for a few reasons:
- Data Discovery is usually an intended outcome of a BI project, but it also leads to changes in requirements as per the original SOW.
- 2 out of 3 deliverables are at 100% or greater EV to expectations, and the third is at 90%. A sizable change request could put the entire project at risk. Is it worth it?
- Different business users, having different roles and responsibilities, might have different opinions on the importance of having Deliverable #3 at 100% EV to expectations.
Perhaps the best outcome is to accept the shortcomings of Deliverable #3. The new findings can be added to a “parking lot” of items for the next round of development. Realistically, the discoveries that led to Deliverable #3 falling short of expectations are in fact value derived from the project. Previously unknown details about the data were discovered during the project.
Convincing business sponsors to postpone re-work can often be a challenging task requiring a project manager to communicate effectively. Ideally, a BI project team should always be prepared for findings that will require changes to the SOW. Expectations for project schedules, budget, and the risks of change will be different for every project and must be evaluated on a case-by-case basis. Knowing, communicating, and understanding the needs of your stakeholders and sponsors will help guide the decision to either re-work a deliverable or else postpone changes until a later time.