Big Data, King of the Terabytes!

[#1]

To celebrate Halloween last month, I went to my friend’s home, a database applications programmer that enjoys referring to himself as a ”Brogrammer,” to watch scary movies like “Godzilla, King of the Monsters!” and help hand out candy to the likes of Spider-Man, princesses, and little Elmos to name a few that rang the doorbell with a proposal of a trick-or-treat. Hey Bro, I asked my Brogrammer friend, there’s so much buzz lately about Big Data, what do you make of it? His response was rather thought provoking.

“Todd, there’s a monster in our presence, and I’m not referring to my son’s stuffed Ugly Doll that I’m holding in my hands. This is far more alarming, and like many creatures, the more it takes-in the more unruly it gets. And this toothy beast is constantly getting bigger, 24/7 as it gobbles down on all shapes and forms of digital information. Both structured and unstructured bytes of data, including digital exhaust data—data that’s created as a by-product through the use of billions of consumer devices and social media sites around the world— contribute to the gigantic size of Big Data, a data set that’s grown so large that it has become awkward to capture, manage, or process using relational databases and desktop statistics/visualization packages, requiring instead massively parallel software running on tens, hundreds, or even thousands of servers to manage Big Data, King of the Terabytes!”

Hmm, OK, so Big Data consists largely of unstructured data like emails, voicemails, pictures and video files, etc., and refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze. Many feel that a great deal of Big Data should simply be destroyed for the sake of privacy, while many others feel that it should be studied to better serve customers and optimize operations. What do you think?

They are so wrong. Big Data should not be destroyed, it should be studied

“Privacy concerns or the lack thereof, are nothing new. So what's different now?  The difference is Big Data, and with it are tremendous opportunities for companies to not only improve their bottom line, but also improve upon quality of life. However, gaining insight from data becomes challenging as it grows extremely large.

Case in point, in the healthcare industry there’s a lot of data that’s not tapped and that also includes the analytics against that data. For example, in the medical field the amount of data that’s in unstructured textual form is huge in terms of not only what the doctor writes down about a patient, such as age, gender, height, weight, medical problems, but also about what the doctor prescribes, things like medication, lab work, tests, and other various procedures, including some that are invasive—many possibly inappropriate and costly. But, if it’s not captured and analyzed we don’t really know how effective the doctor is doing their job. Today, because of Big Data and analytics, we have the ability to take a look at some of these things and with far greater accuracy determine whether it’s the right thing to do for a particular patient, or determine if some of these costly procedures are really necessary.”

You have your fear, which might become reality; and you have Big Data, which is reality

OK, so now this whole Big Data, complexity, velocity, variety, and volume thing is beginning to finally make sense to me.

In today’s enterprise, information is perhaps the most important asset. As you capture trillions of bytes of information you not only struggle with capturing and warehousing this growing amount of valuable digital information, but also with how to analyze and derive value for innovation, and growth. However, overburdened IT systems are having increasing difficulty managing these workloads. This is where the future of IT is going, and this is where companies like HP is investing a lot of resources for unlocking trapped information for better insight, foresight, and decision making. Does that sound right?

“I think he’s got it,” says my Brogrammer friend. “Now go answer the door. It’s your turn to pass out candy.”

Information Optimization—Getting the Most from Your Data

In organizations today, just 15 percent of data is found in structured formats that computers can understand. The remaining 85 percent consists of "Human Information," data found in video, audio, emails, texts, social media, and other sources. Is your stored information working for you by helping you maximize business opportunities? How easily can you demonstrate regulatory compliance, or find necessary documentation for legal action? HP Information Optimization solutions include the leading hardware, software, services and partnerships to help you extract 100 percent maximum value from your information environment.

Autonomy, an HP company knows how to store data, archive it, access it, find it, analyze it, report on it and return it to where it should be kept. And, a huge bonus is that it plugs right into cloud systems. When combined with HP’s Vertica analytic database, HP has a very complete solution to Big Data--Combining Vertica’s columnar, lightning-fast analytic database with Autonomy’s unstructured text analytics technology results in the Big Data platform that enterprises and public agencies require to analyze unstructured, semi-structured and structured information in a consistent, uniform way.

Instant-On

To realize the power of the Instant-On Enterprise, you must be able to extract actionable enterprise insight from today's staggering volumes of data and deliver real time access to information and resources. The Instant-On Enterprise redefines how organizations gather, store and analyze data to enhance the decision-making process across your company. With HP Information Optimization you can effectively transform your data into actionable insights to deliver better products and services to your customers by changing how your informational assets are captured, stored and analyzed. HP Enterprise Services can get you started on your Instant-On journey. HP offers a fundamental set of solutions focused on helping you transform your enterprise–Application Transformation, Converged Infrastructure, Enterprise Security, Information Optimization, HP Hybrid Delivery and Cloud.

A key component of the value proposition of Big Data is analysis. What can analysis of Big Data do to change your business?