Is Your Company Sidestepping the Issue of BI Data Chaos?
Who is curating the business intelligence data at your organization, and is that process effective in preventing data chaos? Can you confidently answer this question?
Increasingly, there are many interesting, valuable sources of data outside an organization. In many cases, end users leverage and analyze these data sources and make decisions without governance. The result: data chaos. I brought up this topic at one of my Friday #BIWisdom tweetchats. Attendees, who were BI users, consultants and vendors, had a lot to say about data chaos.
One attendee tweeted that end-user data preparation may cause greater data chaos. Someone added, "I even wonder to what extent geographical issues play a part in data chaos." Another chimed in with this tweet: "Those who think that end-user data prep is less than critical don't understand how messed up results get if data is bad." Someone else added, "My first self-empowered data visualization ended horribly because my data was not aligned properly." Summing up the situation, one of the group tweeted, "with the proliferation of ungoverned data in an organization, sometimes there is no means of even knowing where all the data is."
We plowed forward with questions you may want to consider for your organization. Here are some insightful opinions and conclusions of the #BIWisdom tribe:
Q: Can we enable data diversity while managing inconsistencies and chaos, or are these two goals contradictory?
A: Without a quick and simple gateway for incoming data, people will take the shortest path, and that leads to "data chaos."
Q: Is it correct that any data that improves perspective should be part of the mix?
A: "I would elaborate that it's any data that improves perspective for a concrete ROI. Sometimes the effort and cost are not worth it."
Q: How can we simplify the intersection between data science skills and tech?
A: "It depends on the maturity of the organization and collaboration between business and IT."
Q: Governance should be a business function executed by IT. This is often not the case. How can the business be more engaged in this process?
A: "The business should manage expectations, qualify data sources and give an indication of appropriate usage." "There needs to be a close partnership between IT and the BI team that educates users on the impact of their actions with data."
But that kind of partnering connection between IT and the BI team is not evident in most organizations. "Do executives recognize how important this is?" I asked. "How can organizations elevate this as a strategic issue?"
Who is responsible for identifying and quantifying the "value gap" in the use, quality and strategic value of data? Whether BI ownership belongs with IT or with the business has long been an issue, and it was clear in the tweeted comments that day that its still an issue:
- "IT now looks for solutions, not tools."
- "In my belief system, the business owns this, but it must include a strong partnership with IT as a key member."
- " I disagree that this is governed by IT. The business needs to make changes on the fly. This is more about simplicity, not snappy UI/UX."
- "BI should not be governed by IT, simply executed by them as a provider of a service. The business is critical in the feedback loop for two-way communication."
- "The problem here is that what the IT team thinks is easy updates is more like DBA Hell slow processes and real chaos."
- "People talking from cross purposes can take an age to solve a problem. A simple process that both IT and the business can support is needed."
- "Until we routinely get the chief data officer reporting directly to the CEO in large organizations, then the business thinks data chaos is an IT issue."
- "The enterprise and solution architects need to solve the process and people problems first before the technology."
"So, whats the solution", I asked the group. "What have you seen working successfully?"
One of the attendees tweeted that he likes to support information management through a governing body. He described a governing body comprised of (A) a steering committee responsible the vision, an operating committee responsible for the strategy and a working team responsible for execution. But someone else commented that committees slow down a process and there should be a single person responsible for the area of data in a department.
Someone else tweeted about a successful strategy he has seen: getting the data producers and the data consumers together in a room to discuss and understand how data is used. Another attendee claimed success with a governing partnership at the VP or Director level with a key leader from the business and one from IT. And another person tweeted he had seen jointly owned initiatives by IT, Finance and Operations be effective.
And someone observed that success doesnt usually happen until you make it a component of compensation for all C-suite members.
But the solution to data chaos (no matter what the solution is) involves disruptive change. And I reminded the #BIWisdom tribe that a lot of organizations (especially publicly held companies) arent willing to engage in disruptive change even if its the right thing to do.
A tribe member tweeted that averting data chaos may require a crisis to spur change. Then a burning platform can be the rallying cry an organization needs to engage the right people at the right time to make a difference.
Bottom line: Can we enable data diversity while managing inconsistencies and data chaos, or are these two goals contradictory? Its a tough balancing act. The solution must be free of politics and a desire to control end users.
Should the process to avert data chaos be top-down or bottom-up? I believe both approaches are necessary to succeed strategically; without both, the process wont be valued in the organizations culture. Ah, the culture theres part of the issue. A collaborative culture is necessary for the solution. There must be a foundation of trust and transparency. Some technologies can help to avert data chaos. But this issue is not a technology problem. Data governance or data chaos is a cultural and communication issue.
Howard Dresner is president, founder and chief research officer at Dresner Advisory Services, LLC, an independent advisory firm. He is one of the foremost thought leaders in Business Intelligence and Performance Management, having coined the term “Business Intelligence” in 1989. He has published two books on the subject, The Performance Management Revolution — Business Results through Insight and Action, and Profiles in Performance — Business Intelligence Journeys and the Roadmap for Change. He hosts a weekly tweet chat (#BIWisdom) on Twitter each Friday. Prior to Dresner Advisory Services, Howard served as chief strategy officer at Hyperion Solutions and was a research fellow at Gartner, where he led its Business Intelligence research practice for 13 years.