Researching shifts in computing in a data abundant world
I mentioned that I was giving a presentation this week at the New Horizons Forum at the AIAA conference. Since it may provide some useful insight about the research underway at HP labs in a larger context, here is the content of one slide from that presentation:
1 datum is a point
2 data are a line
3 data are a trend
100 data are a picture
Having sensors to generate the data that fuels a more proactive business is important, but there is more to sensing than the sensors and the data collected. A holistic ecosystem view is needed. Unfortunately, this means that the tools of today may not be up to the tasks required.
You may have heard about HP’s efforts to place a million node sensor network in the ground for Shell, gathering seismic information. Traditionally, this kind of information was just a flash of perspective taken in the dark from a few locations. Instead, this sensing effort with Shell generated a much more fine-grained view, taken from a myriad of angles, to understand in-depth what was underground.
In order to do implement the system, HP not only had to invent the sensors (relatively cheap and yet very sensitive MEMS devices), but we also create the networking and management techniques to make it useful. Building upon what we’ve learned, we’ve been researching whole new approaches to information storage and computation that will be required to generate value from massive amounts of information.
HP has many of the foundational patents on memristor devices and sensing techniques and we should soon see the shift in storage and computing that the implementation of these techniques should enable. The whole concept of computing will likely need to bow to the onslaught of information from sensing and the related metadata, changing how information is transferred within the computing environment -- shifting from computing on bits to analyzing information in graphs on highly parallelizes computing platforms: Cog Ex Machina
In addition, research is underway to understand how information can be analyzed, automated and displayed. New techniques can be applied to focus attention on the areas needing the creativity that people can provide.
In the marketplace, last year was the year of Big Data as a buzzword with its primary focus on generating insight from the massive amounts of information being collected. Frankly, that will not be enough for the future envisioned – we need to shift the focus to time-to-action, not insight and that is what many of our research efforts underway will enable.
How will collaboration in a machine-to-machine enabled world change?
Distributed networks and collaboration are capturing consumers and individuals experiences. There has been an increase in the tools that can be used to mine all those connections (Big Data). The individual is becoming more relevant at the crossroads of change, whether it is the redistribution of awareness or action. People are becoming more active in trying to shift their environment, some see this new wave of capability as very empowering.
Social technologies are becoming an important lever to increase influence -- permeating our work and personal lives. They are enabling individuals to recognize others like them, as well as those who are quite different in how they view the world.
Naturally there is a bit of work involved in finding those others, since many are content to interact with those similar to themselves. The information is out there though for those who want to diversify their views. Individuals are beginning to recognize that they can influence their future and how value is perceived.
In addition to all this internal capability, externally, our home’s lights, heating and security systems can be controlled via smartphone from anywhere in the world. We’re living in a world where almost any new appliance purchased can be (or soon will be) connected to the Internet. We can control them or they can even interact with each other. Our social network will soon start to include machines as well as people, even if all they can do today is prepare our environment for more effective social interaction.
There have been demonstrations of smart pill bottles that tell you when and how much medicine to take, cars that tell you when you are too drunk to drive or that help you brake more effectively and nano-medicine measurement techniques are coming that will measure our health by taking readings from the things we touch and share those readings.
We can be thankful for these possibilities… It is a world where the early adopters may already live. It does make you think about how things will change.
OpenFlow based Internet2 research platform coming soon...
The consortium hopes to eventually deliver an enhanced network that can handle the transfer of large data sets in response to the abundance of data today. Having this high speed network crossing the US will provide a solid test bed for further enhancement to future networking needs of those initially involved as well as everyone else in the future – just like the original Internet.
Can technologists be measured by the value of their network?
It’s clear we live in a technical world that is undergoing constant change and one of the key skills technologists have to have is understanding, managing and keeping up with change. Technologists can either do this directly or by having access to a large, inter-disciplinary network of individuals with diverse skills.
Organizations don’t always realize that when valued technologists leave, the network they have cultivated can dissolve and various branches (that no one knew existed) can die. People are not fungible. Unlike the formal organizational structure, these interdisciplinary networks are never written down. Technology needs to have a role in understanding and even documenting these relationships.
The skills of collaborating and supporting others should be one of the key selection criteria for technologists within larger organizations. The importance of this skill is right up there with analytic and creative skills. Its importance is another example of why diverse skills within an organization are so critical to facilitate an innovative culture. It is probably not limited to technologists either, here is a post Internet Evolution that shows team building right at the center of the CIO skillset.
The Open Mind blog points out that a variety of techniques are effective:
Some people question, others observe or experiment but networking is the one skill I see that pulls the organization together into action.
That makes me ask as we start to look at an individual and their impact on the organization: What is the depth and breadth of the network they developed?
Play changes for IT on the second half of the board
One slide that was used in HP Discover last week quite a bit was this one:
It shows how technology has shifted since the dawn of the Information Technology. These changes are not likely to slow down because it is all fueled by exponential technology growth.
The best illustration that I can give you of what ahead is a little story that Ray Kurzweil used in his book called the Singularity is Near.
It is about the unimaginable change that is possible when driven by exponential growth. The story starts with the man who invented chess. When he showed the king of India the game, the king was so entertained and excited by the game that he told the man he’d give him anything he asked for – within reason.
The man made what appeared to be a simple request. He asked that every year for the next 64 years (the number of squares on a chess board) a few grains of rice in the following manner: the king was to provide a single grain of rice on the first chess square and double it every following year.
The king quickly agreed.
The first year the inventor received 1 grain, the second 2, the third 4… It doesn’t get interesting until you cross over into the 2nd half of the board.
On the 23rd square we are talking about 8M grains. A still reasonable amount of rice, that can be delivered by a small field of rice. At the next square, when crossing over to the 2nd half, the king finally took notice, because now it would start impacting his grain inventory. The king realized by the time they would reach the end of the board, it would have required enough rice to cover all of India one meter thick with rice. He’d been had and the inventor’s head was soon cut off and the rice deliveries were no longer a problem.
I bring this up because all these exponential trends that we’ve been taking advantage of in IT, like Moore’s law, Edholm’s law…, are now reaching into the 2nd half of the board. We’re the ones who need to understand and take advantage of the change since it is quite different than what we’ve seen to date.
How many of you have already felt the constraints of your own thinking getting in the way of technology adoption? I know I for one need to take a step back every once in a while and say “what does this really mean?”
And I ask myself “What can be done with this abundance of capability that has never done before?”
We are entering into a different world where there is an abundance of data – with all the sensors and mobile devices… We don’t worry as much about if the data is available, but more about what we can do with it. For those people who believe that data is king, it can be a rude awakening when they realize that in a world of abundant data, having more of it is worthless.
We don’t worry as much about if we can transport the data to the processing location. The networking is typically there, although it may still cost more than we wish.
With cloud computing, we have the resources to crunch all that data into something useful.
Additionally, our access to software capabilities is more than we’ve ever seen before as instantiated in the phase – there’s an app for that. For businesses that may be SaaS, Open Source, COTS…
For most businesses though the systems were designed with a very constrained view of the world. They were based on scarcity of data, computing… and it is time to take a step back and really look at that portfolio of applications for what they are really good at and how they add value.
