# On the Value of Data

Data has become increasingly intertwined with our daily lives as more companies collect, analyze, and utilize it—and its use is growing exponentially. Data is _everywhere_. IoT is opening up new possibilities in how easily and efficiently we can collect it. Breakthroughs in machine learning are creating demand for larger data sets to improve the accuracy of predictive models. It's clear that at least some data has value.

In 2006, Clive Humby—a mathematician, entrepreneur, and business strategist—said, "Data is the new oil." Maybe you've heard it before. It has become a kind of popular memetic aphorism. The meaning of this quote was expanded on later by Michael Palmer, vice president of the Association of National Advertisers. He added, "Data is just like crude. It's valuable, but if unrefined it cannot really be used." Whether or not you believe this to be true, it's difficult to argue that data has _no_ value.

## Why is data valuable?

The answer is easy: _Because the old oil is valuable! The new oil must be valuable!_

Not a good enough answer? Okay. There's more to it than that. So, what do we even mean when we say "valuable"? Money? That's certainly the most common way to measure the value of data in many companies.

<p style="font-size: 0.9rem;font-style: italic;"><img style="display: block;" src="https://live.staticflickr.com/6211/6355836713_7ea15f733f_b.jpg" alt="Money"><a href="https://www.flickr.com/photos/68751915@N05/6355836713" target="_blank" rel="noopener noreferrer">"Money"</a><span> by <a href="https://www.flickr.com/photos/68751915@N05" target="_blank" rel="noopener noreferrer">401(K) 2013</a></span> is licensed under <a href="https://creativecommons.org/licenses/by-sa/2.0/&atype=html" style="margin-right: 5px;" target="_blank" rel="noopener noreferrer">CC BY-SA 2.0</a></p>

In the business world, data is often used in attempts to increase revenue and decrease costs. Data doesn't _always_ have intrinsic value. A company could have mountains of data that's worth nothing (or worse, its value could be a monetary loss depending on how much effort was involved in accumulating it), but when the data does have some insight to offer, it gains monetary value.

- In the U.S. alone, according to one [big data market revenue forecast](https://www.statista.com/statistics/254266/global-big-data-market-forecast/), the market value is expected to increase from the current $70 billion to $103 billion by 2027.
- In a [global forecast](https://www.marketsandmarkets.com/Market-Reports/big-data-market-1068.html), the big data market is expected to climb from its current estimate of $163 billion to $273 billion by 2026.
- Based on a [survey from 2015](http://barc-research.com/big-data-analysis-shown-to-increase-revenues-and-reduce-costs/), an estimated 40% of companies worldwide were analyzing big data, and they reported:
- An average increase in revenue by 8%.
- An average reduction in costs by 10%.

These examples only scratch the surface of the monetary value of data, but this value is only an **effect**, not a **cause**.

The true value of data is **determined by its ability to solve problems through insight and how people can use it**. Valuable data solves problems. For example, from the same 2015 survey (the last example above), the top benefits of analyzing data were:

- Better strategic decisions.
- Improved control of operational processes.
- Better understanding of customers.

These benefits almost certainly led to some of the increased revenue we saw in the survey, but, more importantly, they improved quality of life. A better understanding of customers leads to happier customers. Better strategies and control of operational processes can result in less stressful work environments for employees. Even if a company is only interested in the money at the end of the process, the data itself must possess non-monetary value to get there. This is the more profound value of data and it's the _cause_ of data having monetary value.

Now, let's move away from business and money to focus solely on the underlying value. The concept of data has been around for a _long_ time. From tally marks on stones to rows of servers full of data, a lot has changed, but the core of what makes data valuable hasn't. Sciencific research is always a great place to look to see the impace of the value of data.

<p style="font-size: 0.9rem;font-style: italic;"><img style="display: block;" src="https://live.staticflickr.com/5047/5370298016_d95b35977d_b.jpg" alt="Maternal Health in Developing Countries"><a href="https://www.flickr.com/photos/35483578@N03/5370298016" target="_blank" rel="noopener noreferrer">"Maternal Health in Developing Countries"</a><span> by <a href="https://www.flickr.com/photos/35483578@N03" target="_blank" rel="noopener noreferrer">United Nations Photo</a></span> is licensed under <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/&atype=html" style="margin-right: 5px;" target="_blank" rel="noopener noreferrer">CC BY-NC-ND 2.0</a></p>

In 2006, the US maternal mortality rate average was 13 deaths per 100,000 births. [In the same year, California started the California Maternal Quality Care Collaborative](https://www.mitre.org/publications/project-stories/can-data-modeling-and-analytics-help-reduce-pregnancy-related-deaths), which focuses on decreasing these numbers using data. They analyzed data on contributing factors and used what they found to create toolkits consisting of best practices for given scenarios. Since then, the numbers in the US as a whole have climbed, while decreasing in California. In the first 7 years, they lowered the mortality rate by 55%.

| Year | US Total | California |
| - | - | - |
| 2006 | 13.3 | 16.9 |
| 2007 | 12.7 | 11.1 |
| 2008 | 15.5 | 14.0 |
| 2009 | 16.6 | 11.6 |
| 2010 | 16.9 | 9.2 |
| 2011 | 19.3 | 7.4 |
| 2012 | 19.9 | 6.2 |
| 2013 | 22.0 | 7.3 |

<p style="font-size: 0.9rem;font-style: italic;"><img style="display: block;" src="https://live.staticflickr.com/7507/30174084905_4a4890506e_b.jpg" alt="Kras-Driven Lung Cancer"><a href="https://www.flickr.com/photos/132318516@N08/30174084905" target="_blank" rel="noopener noreferrer">"Kras-Driven Lung Cancer"</a><span> by <a href="https://www.flickr.com/photos/132318516@N08" target="_blank" rel="noopener noreferrer">National Institutes of Health (NIH)</a></span> is licensed under <a href="https://creativecommons.org/licenses/by-nc/2.0/&atype=html" style="margin-right: 5px;" target="_blank" rel="noopener noreferrer">CC BY-NC 2.0</a></p>

Another example is cancer detection. You might have heard or read about this. It has done its rounds in the media, and for good reason. Good data paired with machine learning can greatly improve the chances of detecting cancer early.

- This [2020 article from Nature](https://www.nature.com/articles/d41586-020-03157-9) describes the impact and success they've seen so far with lung cancer detection:
- A decrease in mortality by 20-30%
- Their system correctly identifies early-stage lung cancer 94% of the time (outperforming a panel of 6 senior radiologists)
- Another [2020 article from Nature](https://www.nature.com/articles/d41586-020-00847-2) discusses a wider application of cancer in general. They've trained a model on 60,000 tumor samples:
- Recognizes around 150 types of cancer
- The model found 79.2% of lesions vs 80.7% found by radiologists with at least 10 years of experience, but the model's numbers are expected to increase in the future

## Why is quality data important?

<p style="font-size: 0.9rem;font-style: italic;"><img style="display: block;" src="https://live.staticflickr.com/221/481220595_8ccf2e6879_b.jpg" alt="quality¿"><a href="https://www.flickr.com/photos/11693777@N00/481220595" target="_blank" rel="noopener noreferrer">"quality¿"</a><span> by <a href="https://www.flickr.com/photos/11693777@N00" target="_blank" rel="noopener noreferrer">MrVJTod</a></span> is licensed under <a href="https://creativecommons.org/licenses/by-sa/2.0/&atype=html" style="margin-right: 5px;" target="_blank" rel="noopener noreferrer">CC BY-SA 2.0</a></p>

Before you can solve problems or answer questions with data, you must first validate its quality. This will ultimately determine whether or not it has value. Some of the widely accepted measures of data quality are:

- Accuracy - one of the most critical aspects. Without accurate data, any answers or findings aren't accurate either.
- Completeness - should provide enough attributes to satisfy its use requirements.
- Consistency - values for a given attribute should be consistent across all systems using it.
- Timeliness - should be as up-to-date as possible. The further it falls behind, the more likely it will become inaccurate.
- Uniqueness - shouldn't have unnecessary exact duplicates _or_ non-exact match duplicates (see: Consistency—there could be overlap here)
- Validity - all values of a given attribute should use the same format.

Without quality data, the results of an analysis aren't likely to be accurate or helpful. This can lead to many problems. A few of the most obvious would be:

- Huge monetary losses
- Poor quality research results lead to the spread of incorrect or false information (which could affect an almost infinite number of aspects of our lives)
- Or, worst-case scenario, if you're working in a field that deals with human life or safety, it could lead to death or injury.

## The future of data

<p style="font-size: 0.9rem;font-style: italic;"><img style="display: block;" src="https://live.staticflickr.com/405/20010071236_f31f15a492_b.jpg" alt="Futuristic Utopian city"><a href="https://www.flickr.com/photos/135032189@N08/20010071236" target="_blank" rel="noopener noreferrer">"Futuristic Utopian city"</a><span> by <a href="https://www.flickr.com/photos/135032189@N08" target="_blank" rel="noopener noreferrer">Futuristic Society</a></span> is licensed under <a href="https://creativecommons.org/publicdomain/mark/1.0/&atype=html" style="margin-right: 5px;" target="_blank" rel="noopener noreferrer">CC PDM 1.0</a></p>

Data can offer value in a myriad of ways. With the continual growth and evolution of data collection, analytics, IoT, and machine learning, the ways in which data proves its value will certainly do the same. We will undoubtedly continue to see improvements to our quality of life. However, as data is involved in more and more aspects of our lives, it will become increasingly important to ensure that our data is of the best quality.

As an optimistic futurist, I hope that more people can acknowledge the potential of data and use it responsibly, conscientously, and to benefit humanity as a whole. Of course, as with any power, tool, breakthrough, or knowledge, there will always be people that use it for nefarious purposes. So far, the impact of data on the world looks like a net positive. We just need to do our part to make sure it stays that way.