Data is the lifeblood of modern companies and companies are creating more data than ever before. Hospitals now rely on electronic medical records to record and monitor interactions with patients, identify risk factors, and recommend treatments. Retailers utilize purchase data to recommend products, segment customers, and target promotions. Manufacturers leverage data to monitor product quality and proactively intervene before breakdowns occur. The list goes on. Data growth in the US averaged 31.9% annually from 2010-2018. According to Market Research from IDC, more data will be created this year and next than in the entire prior history of computers.

Data can be your competitive advantage; it can also be your blind spot.

Data growth is being driven by company needs and opportunities. As more economic activity migrates online, remote, and into mobile devices, data utilization is the most effective way your organization can understand and adapt to what is happening, both internally and externally.

This trend has been accelerated by the coronavirus pandemic. Global Workplace Analytics estimates that 25-30% of people will work from home multiple days per week after the pandemic ends. Prior to the pandemic, online sales accounted for only 6.3% of total grocery purchases. Now, nearly 40% of consumers say they plan to buy groceries online in the next 12 months. That seems like an underestimate. The time to capture value from data is now.

The CorrDyn Data Journey Map includes four Data Value Buckets where companies can identify ways to use their data more effectively:

1. Increase revenue: Data availability can allow for new products and services that were previously unachievable by your business or any business. 
   
   
2. Decrease cost: Data can be utilized to reduce costs through process automation, portfolio optimization, efficient resource allocation, and waste identification.
   
   
3. Increase Accountability: Data can provide the transparency necessary to monitor your business as a manager and drive actionable improvements on the front lines.
   
   
4. Mitigate risk: Data can create opportunities to proactively identify risks to equipment, enterprise infrastructure, customer retention, and business processes.

Yet for all these opportunities to add value, most companies fail to capture meaningful value from their data. Our Journey Map includes three value drivers that determine a company’s capacity to capture value from their data:

1. Availability: Companies fail to make their data available to the people that need it. 
   
   
2. Actionability: Companies fail to make their data actionable for the people who receive it.
   
   
3. Credibility: Companies fail to make their data credible for people to use in decision making.

In the sections that follow, I diagnose the challenges underlying each category of failure.

Data Must Be A vailable

• Creating the appropriate structure and format for analysis: For analysts, this might mean pushing the data into a data warehouse that can be queried and manipulated on demand. Raw data might live in multiple formats, including JSON, CSV, HTML, XML, images, PDFs, and log files (among many others). Processes need to be created to enable integration of different data formats and extraction of the most useful information from those formats. Most companies default to using only the data that is easiest to read.
  
  
• Enabling access to the data for employees in different locations and using different tools: Some employees might want direct access to data using a query language like SQL or might need an interface to export lists to spreadsheets. Others might need a simple dashboard to monitor metrics. Internal systems need to allow data to meet employees where they can leverage it most efficiently, according to their needs, skills, location, device, and bandwidth.
  
  
• Ensuring the level of granularity of data is sufficient for business requirements: The granularity of data is the lowest level of detail to which the data can be attributed. If your marketing department needs individual customer-level insights, attributable to specific customers, but the only data available is aggregated daily information about your website from Google Analytics, then you have a data granularity problem.
  
  
• Providing transparency about the existence, source and lineage of data: Analysts can’t hit what they can’t see. In order to convert data into actionable insights, companies need to empower people to utilize available data assets. Word-of-mouth explanation and emailed spreadsheets will not suffice for ongoing integration of data into business processes.

Once data is available, companies have another set of challenges to make that data useful.

Data Must Be Actionable

Integrated + Synthesized + Timely = Actionable

‍

 

We define actionable data as data that is integrated, synthesized and timely.

Data integration involves combining the highest value marketing, sales, customer support, web, technology, operations, logistics, and HR data into a single source of truth for the enterprise. This might mean extracting information from on-premise databases, cloud infrastructure, and software-as-a-service applications. Regardless of the source, data from each location will have to be manipulated into a consistent format that enables insights to be aggregated from all the sources simultaneously. Your business outcomes dictate the nature of these transformations.

Data synthesis requires engagement with the data to generate insight. This could be from an executive, business intelligence (BI) analyst, data scientist, or anyone with business context who can interpret the data for a purpose. An executive might use Excel or PowerPoint, a BI Analyst might explore data using a data visualization tool like Tableau or Looker, and a data scientist might explore using Python and Jupyter Notebooks. Regardless, the goal is to extract insights from the data and present those insights in a way that clarifies action. Insights can be as simple as “call a customer who hasn’t logged in” or as complicated as “prioritize our leads using web traffic data and allocate them to different salespeople based on their behavior.”

Data timeliness means that insights gleaned from integration and synthesis are generated at the right place and right time for action to be taken. Backward-looking insights can be interesting, but are usually not as useful as seeing the current picture. The entire data pipeline needs to account for the speed with which information needs to be turned into action. Customers who haven’t logged in need to be called before they churn. Leads requiring allocation have to be scored as they come into the sales funnel.

If you have actionable data, you are most of the way there, but there is a critical element that even companies with analytics teams get wrong: data has to be credible or it will be ignored.

Data Must Be Credible

‍

Data credibility consists of data that is:

• Reliable: Data pipelines rarely break down and they produce consistent results.
  
  
• Clearly defined: Decision-makers understand what the data means and they can articulate the meaning to others. 
  
  
• Culturally important: Leaders at the company have established a culture of making smart decisions using data and other decision-makers follow suit. In the words of Ed Deming, "In God we trust, all others bring data."

If your company has available, actionable, and credible data, then you are already a top company on the CorrDyn Data Maturity Scale. The final and most critical step is to be able to consistently generate positive ROI from your data projects.

Positive ROI Data Projects

Data projects are hard to manage because they do not fit neatly into a software engineering, business project, or R&D framework that companies typically use to manage their investments.  Poorly designed data projects can have negative ROI for four reasons:

‍

1. High Cost: Between expensive data infrastructure and expensive data talent, it is easy to rack up costs relatively quickly. 
   
   
2. Uncertain Timelines: The experimental nature of some data projects, especially machine learning initiatives, can lead to wasted effort and an inability to determine when a project is “complete”.
   
   
3. Low Return: Unclear or changing business requirements, an uncertain path from data to business value, and bllind optimism about availability, actionability, and credibility of data can all lead to returns on data projects that do not meet company expectations.
   
   
4. Unreliable Engineering: Data developers do not always embody software engineering best practices. Thus, data projects can have architectural flaws that lead to high maintenance costs.

Well-designed data projects have five drivers that ensure their success:

1. Strong executive buy-in: Leaders know their goals and do not deviate from those goals.
   
   
2. A clear path to value: Non-data professional staff can understand how available data that the company collects can be converted into actionable insights that are timely.
   
   
3. Transparent project management: Data leaders have the ability to convert business requirements into a project plan with milestones that everyone can evaluate. 
   
   
4. Judicious architecture for the needs of the business: By scoping the project strictly to business requirements, and scaling over time, data infrastructure costs can be minimized.
   
   
5. Engineering for low long-term maintenance: Data team members can develop data pipelines and processes that work reliably without large overhead costs.

CorrDyn’s Five Drivers of Successful Data Projects are also the key success factors for all CorrDyn data science and data engineering engagements.

If your company is relatively early in its data journey, a partner like CorrDyn can help guide you on the fastest path to value. If you are relatively far along, a partner like CorrDyn can enable your organization to achieve value from your data without building an entire department to generate that value. Wherever you are along the Data Journey Map, CorrDyn can help you achieve your objectives by providing expertise as needed, without the overhead.

Your company’s data projects can be successful if you understand the roadmap and keep your eye out for the roadblocks. Be mindful of your company’s data journey. Your company’s future growth and profitability likely depend on its success.

Keep building.