The use of data to run the business, as well as to enhance user engagement and improve the customer experience, has gradually become an essential discipline in business strategy and operations.
Many businesses recognize the importance of data-driven decisions, however how to derive value from the data is a different challenge. Fundamentally, organizations face the problem of aggregating data from numerous sources, often in silos,accommodating varied data elements, and integrating them in unified formats that can support easy access and immediate extraction. While companies want to be metrics driven, a great number of businesses don’t know how to use the data and what are the right questions to ask. At Structure Data 2016, data science and data analysis experts discussed the data-driven revolution, how it is becoming mainstream in most industries, and the challenges businesses face today.
Not all functional departments in the company know what to do with the data. In most businesses, Sales can clearly define metrics to measure success along the selling pipeline. But understanding customer attrition is harder to figure out, even when you have the user engagement data across multiple channels and throughout the customer life cycle.
Chris Neumann of DataHero, Dan Wagner of Civis Analytics and Tom Krazit of Structure Events talked about the challenges companies face with getting the most of data analysis. Many companies today are struggling with how to derive value from the data. There is a distance between the underlining data and technology assets the organization has collected and the ability to quickly answer important basic business questions. For example, it is difficult to answer simple questions of “what is going on in the business?”, “How to get the big-picture across all business units?”, “Can the company’s top leaders and decision-making personnel get a timely snapshot of situational insight of the business?”, “What are the reasons for what is happening in the business?”, “What should the business do?”, etc.
The challenges can be divided into three major areas:
- Technology obstacles – Can the data scientists build an environment where all the data is unified and accessible in real time? Can data be produced as needed? Can insights be derived across the business units?
- Organizational structure can create obstacles to access data: who can access what, when and how often? What are the processes to obtain the data, with minimal or no data segregation?
- Cultural issues – In many organizations, the different business units have their own data scientists and analysts, where a single business unit cannot get answers to strategic or high level business questions.
Becoming data savvy means that various non-technical departments in the organization and functional subject experts need user-friendly tools and access to relevant data and metrics. Data analytics tools need to become more obtainable, easy to use and more intelligent.
All of these challenges present opportunities and the data driven market has hundreds of players with various software solutions. Over-promising is also an obstacle: utilizing Hadoop, an open-source software framework for storing data and running applications, doesn’t necessarily mean the business can get answers to the basic questions. Dashboards show information, however, the human factor is the key to derive insights and make the right decisions. While Hadoop provides a huge storage solution for all types of data, has enormous processing power and the ability to handle virtually limitless concurrent tasks, the business needs trained and qualified data science staff to ask the right questions and provide sought-after insights that help decision makers.
Using technology will not only make work-streams, business processes and decision making easier, machine learning brings the premise of reinventing new ways to solve challenges. As we move from the Big Data era to the Smart Data domain, the panel experts estimated that within 5 years businesses will use smarter tools. ‘Smart analytics’ coupled with machine learning approaches and the use of artificial intelligence algorithms will revolutionize data analysis.