How Big is Big Data?
We have been talking a lot about Big Data over the past few years. It is believed that as we gather more data we will have better solutions to our business problems. Having stated this, we cannot overlook the fact that today’s data will look minuscule when we start adding sensors to the world. When these sensors take over everything from the aircraft to your cars and from gas stations to vending machines, the volumes of data created will make today’s data look like a dot on a paper. As of today, it is quite difficult to understand the implications of a scenario where every device will be sending in data to a waiting database.
Some IT professionals are of the opinion that one of the outcomes in such scenarios can be that instead of using devices to deliberately measure our activity in the world, perhaps sensors could track our activity and measure it as we move through the world and interact with various sensors – thus, avoiding the need for a specific device devoted to it. What’s happening today is that that the wearable sensors are not trying to make out what’s going on in the environment but are gaining an ability to perceive us instead. Though we are still in the very early stages of the phenomenon it is better to have an understanding before-hand about how pervasive sensor data can become an industry alerting event. Let us, for a moment, think of a time when the Internet of Things becomes more pervasive. What would happen then? Trillions of endpoint devices will be sending information and data to a compute engine. As businesses and humans, our motive would be to make something of that data in real time. This “real time” factor, given today’s relational databases, seems to be the biggest challenge of all. Assume a case of a jet engine sensor sending in information; we cannot wait for hours to process that information. We would want the answers in real time.
Big Data has given way to an era of data intelligence. From data generated by machines to human thought streams, we are accumulating data like never before, so much so that 90 per cent of the data that exists today has been created in the last four years. Each day we create 2.5 quintillion bytes of data which roughly means adding a Google every four days. This is an ongoing process and the rate is only increasing. With these statistics in play, we require better tools for data processing. There are companies like GE, who build large industrial grade types of equipment and want to instrument these large industrial devices and benefit from the data they generate; thus, bettering their efficiency. GE estimates that there will be 17 Billion connected industrial assets by 2025. Today merely ten per cent of those are equipped with sensors. And those which do lack the intelligence which is hoped for in the future, tell us when something goes terribly wrong.
Projects going on in organizations like GE, Zebra and SAP are proofs of concept right now, but it is believed that in time these companies and projects will have a profound effect on how we process Big Data. The irony of the Big Data situation is that more data we encounter more overwhelming it becomes. But the fact remains that large volumes of data give you the best outcomes.
The silver lining is that there are organizations working towards solving these issues before the sensors take over in considerably large numbers. The time to figure out solutions is today before these sensors start transmitting data and the big data onslaught comes hammering.
With such huge and high-quality work happening in the Big Data spectrum, there is a great demand for skilled professionals who understand the nuances of Big Data, sensors and aligned technologies.