The Internet of Things (IOT) will soon produce a massive volume and variety of data at unprecedented velocity. If "Big Data" is the product of the IOT, "Data Science" is it's soul.
Let's define our terms:
Internet of Things (IOT): equipping all physical and organic things in the world with identifying intelligent devices allowing the near real-time collecting and sharing of data between machines and humans. The IOT era has already begun, albeit in it's first primitive stage.
Data Science: the analysis of data creation. May involve machine learning, algorithm design, computer science, modeling, statistics, analytics, math, artificial intelligence and business strategy.
Big Data: the collection, storage, analysis and distribution/access of large data sets. Usually includes data sets with sizes beyond the ability of standard software tools to capture, curate, manage, and process the data within a tolerable elapsed time.
We are in the pre-industrial age of data technology and science used to process and understand data. Yet the early evidence provides hope that we can manage and extract knowledge and wisdom from this data to improve life, business and public services at many levels.
To date, the internet has mostly connected people to information, people to people, and people to business. In the near future, the internet will provide organizations with unprecedented data. The IOT will create an open, global network that connects people, data and machines.
Billions of machines, products and things from the physical and organic world will merge with the digital world allowing near real-time connectivity and analysis. Machines and products (and every physical and organic thing) embedded with sensors and software - connected to other machines, networked systems, and to humans - allows us to cheaply and automatically collect and share data, analyze it and find valuable meaning. Machines and products in the future will have the intelligence to deliver the right information to the right people (or other intelligent machines and networks), any time, to any device. When smart machines and products can communicate, they help us and other machines understand so we can make better decisions, act fast, save time and money, and improve products and services.
The IOT, Data Science and Big Data will combine to create a revolution in the way organizations use technology and processes to collect, store, analyze and distribute any and all data required to operate optimally, improve products and services, save money and increase revenues. Simply put, welcome to the new information age, where we have the potential to radically improve human life (or create a dystopia - a subject for another time).
The IOT will produce gigantic amounts of data. Yet data alone is useless - it needs to be interpreted and turned into information. However, most information has limited value - it needs to be analyzed and turned into knowledge. Knowledge may have varying degrees of value - but it needs specialized manipulation to transform into valuable, actionable insights. Valuable, actionable knowledge has great value for specific domains and actions - yet requires sophisticated, specialized expertise to be transformed into multi-domain, cross-functional wisdom for game changing strategies and durable competitive advantage.
Big data may provide the operating system and special tools to get actionable value out of data, but the soul of the data, the knowledge and wisdom, is the bailiwick of the data scientist.
Big Data Survey of 257 business technology professionals at organizations with 50 or more employees, September 2012.
Organizations desire to understand and predict customer behavior. Data science and analytics can help forge a lasting, profitable relationship between customers and both private and public organizations.
The insights that make this possible are hiding in big data. But how to find and exploit? And how to tell signal from noise?
Oxford and IBM conducted a study "Analytics: The real-world use of big data," showing how over 1,100 business and IT professionals from 95 countries are maximizing the potential of big data. Download the study below.
To compete in a globally-integrated economy, today’s organizations need a comprehensive understanding of markets, customers, products, regulations, competitors, suppliers, employees and more. This understanding demands the effective use of information and analytics. Next to their employees, many companies consider information to be their most valuable and differentiated asset.
Now, with the emergence and expanding adoption of big data, organizations worldwide are discovering new ways to compete and win – transforming themselves to take advantage of the vast array of available information to improve decision-making and performance throughout the enterprise. Not every organization will need to manage for the full spectrum of big data capabilities, but opportunities to utilize new data, technology and analysis techniques exist in almost every industry.
The amount of data your organization produces, collects, stores, analyzes and distributes is growing. But what's the point of amassing all that information if you can't use it to improve production, increase revenues and reduce costs?
Smart businesses are giving people throughout their organizations access to deeper intelligence by marrying their big data and business intelligence efforts into a big data solution. The result is better decisions based on meaningful insights company wide.
What's your strategy for big data analytics?
Data science and business analytics works with both structured and unstructured data. Yet the future belongs to unstructured or semi-structured data from both internal and external sources.
Total Enterprise Data Growth 2005-2015
IDC estimates the volume of digital data will grow 40% to 50% per year. By 2020, IDC predicts the number will have reached 40,000 EB, or 40 Zettabytes (ZB). The world’s information is doubling every two years. By 2020 the world will generate 50 times the amount of information and 75 times the number of information containers.
The massive growth of unstructured or semi-structured data is amazing and has implications for data warehouse / business intelligence / data analytics architecture and database design. The way we capture, store, analyze, and distribute data is transforming. New technologies like deduplication, compression, and analysis tools are lowering costs.
Structured data gives names to each field in a database and defines the relationships between the fields. Unstructured data is usually not stored in a relational database (as traditionally defined) where the data model is relevant to the meaning of the data.
The Internet of Things (equipping all objects in the world with identifying devices), blogs, videos, social media, emails, notes from call centers, and all forms of human and computer to computer communications will soon start to produce massive amounts of unstructured or semi-structured data.
The trick is to create value by extracting the right information from both internal and external data sources. That is what the science of data and art of business analytics needs to learn to extract from larger and larger sets of unstructured data.
See also: http://bit.ly/Sp0IWW