15 Important Big Data Facts for IT Professionals
Keeping track of big data trends, research and statistics gives IT professionals a solid foundation to plan big data projects. Here are 15 important big data facts that every IT professional should know.
Everyone is talking about big data, from the challenges to yearly spending, job creation and even the tools required for big data projects. Many believe big data will help businesses make better decisions — in fact nearly 8 in 10 executives agree or strongly agree to the statement "if we could harness all of our data, we would be a much stronger business."
Keeping track of big data trends, research and statistics gives IT professionals a solid foundation to plan big data projects. Webopedia has compiled this list of important big data facts and statistics that every IT professional should know.
Big Data: How Did We Get Here?
1. How Much Data is There?
Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. This data comes from everywhere: sensors used to gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, and cell phone GPS signals to name a few. (Source: IBM; IBM Big Data)
2. Structured Versus Unstructured Data
In classifying big data, Tata Consultancy Services Limited (TCS) looked at how much of companies' data was structured versus unstructured, as well as how much was generated internally versus externally.
- 51% of data is structured
- 27% of data is unstructured
- 21% of data is semi-structured
A much higher than anticipated percentage of data was not structured – either unstructured or semi-structured and a little less than a quarter of the data was external. (Source: Tata Consultancy Services Limited; The Emerging Big Returns on Big Data)
Big Data Jobs
3. Big Data Generates Jobs
By 2015, 4.4 million IT jobs globally will be created to support big data, generating 1.9 million IT jobs in the United States. Every big data-related role in the U.S. will create employment for three people outside of IT, so over the next four years a total of 6 million jobs in the U.S. will be generated by the information economy. The challenge? There's not enough talent in the industry. (Source: Gartner; Gartner Symposium/ITxpo)
4. The Big Data Talent Shortage
There will be a shortage of talent necessary for organizations to take advantage of big data. By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions. (Source: McKinsey Global Institute; Big data: The next frontier for innovation, competition, and productivity)
5. Rethinking Job Roles and Titles
81% percent of IT leaders and 77% of IT professionals believe there is a significant shortage of workers with the skill required to plan, execute and take advantage of the potential of their organization's data assets. Big data projects might mean rethinking job roles and titles, as well as the non-technical skills needed to make the best use of the data. (Source: TEK Systems; Big Data...The next frontier)
Big Data Challenges and Dilemmas
6. Disparate Systems
66% of IT leaders and 53% of IT professionals claim their data is stored in disparate systems – and these organizations must build new platforms to accommodate these increased data management needs. (Source: TEK Systems; Big Data...The next frontier)
7. Getting Business Value From Big Data
The biggest challenges to getting business value from big data are as much cultural as they are technological. When asked to rate a list of 16 challenges, companies placed an organizational challenge at the top of the list: getting business units to share information across organizational silos. A close second was a technological issue: dealing with what has become known as the three V's of big data: data volume, velocity and variety. The third challenge was determining which data to use for different business decisions. (Source: Tata Consultancy Services Limited; The Emerging Big Returns on Big Data)
8. Data Quality
More than half of IT leaders (57%) and IT professionals (52%) report they don't always know who owns the data. If one doesn't know who owns the data, there is no one to hold accountable for its quality. As different sources and varieties of data are fused together for big data projects, ensuring the accuracy and quality of the data will be critical to success. (Source: TEK Systems; Big Data...The next frontier)
Top Reasons for Big Data Investments
9. Create a Stronger Business
CompTIA found that nearly eight in 10 executives responsible for technical or strategic decisions involving data at their organization, agree that harnessing all of their enterprise data would result in a stronger business. The survey also found that 93% of survey participants say data is critically important to their business; the same percent believe it will be important in 2014 as well.
One of the strongest arguments for investing in data initiatives stems from the following data point: Nearly 8 in 10 executives agree or strongly agree to the statement "if we could harness all of our data, we would be a much stronger business." (Source: CompTIA; Big Data Insights and Opportunities)
10. Better Manage Data
Fewer than 1 in 5 businesses report being exactly where they want to be in managing and using data. Granted, this represents a high bar, but even when including those 'very close' to their target, it still leaves a majority of businesses with significant work to do on the data front. (Source: CompTIA; Big Data Insights and Opportunities)
11. Top 3 Big Data Business Drivers
The top three big data business drivers include:
- Speeding time for operational or analytical workloads (39%)
- Increasing competitive advantage with flexibility of data used in business solutions (34%)
- Business requirements for higher levels of advanced analytics (31%).
(Source: EMA and 9sight Consulting; Big Data: Operationalizing the Buzz)
Big Data Spending and Implementations
12. Big Data Implementations
Big data implementations in production rose from 27% in 2012 to 34.3% in 2013. In addition, 68% of companies are running two or more big data projects as part of their big data initiatives. For companies with an analytics strategy in place, the top business driver was the need to combine sales information into operational analytics (57%). (Source: EMA and 9sight Consulting; Big Data: Operationalizing the Buzz)
13. Big Data Tools
80% of responders said they are already using or planning to use dedicated Big Data tools or architectures in their production environment to cope with the influx of massive amounts of data. 56% of respondents indicated they plan to move existing applications from RDBMS to a NoSQL data store. (Source: Giga Spaces; Big data Survey)
14. Big Data Spending
There's a polarity in spending on Big Data, with a minority of companies spending massive amounts and a larger number spending very little. Some 15% of the surveyed companies with Big Data initiatives spent at least $100 million each on them in 2012, and 7% invested at least $500 million. In contrast, nearly one-quarter (24%) spent less than $2.5 million apiece. (Source: Tata Consultancy Services Limited; The Emerging Big Returns on Big Data)
15. Industries Spending the Most on Big Data
Industries spending the most are telecommunications, travel-related, high tech, and banking; life sciences, retail, and energy/resources companies spend the least. (Source: Tata Consultancy Services Limited; The Emerging Big Returns on Big Data)
Based in Nova Scotia, Canada, Vangie Beal has been covering small business, electronic commerce and Internet technology for more than a decade. You can tweet with her online @AuroraGG.