Flash memory was once viewed as special tool to improve performance or allow for easy transportation of information (e.g., thumb drive – I can’t recall the last time I gave someone a CD, let alone a floppy drive). Now flash memory devices are a standard component of any storage performance strategy.

As the Solid State Drive (SSD) came on the scene, it was used as a plug replacement for spinning media hard drives, providing better performance, but the characteristics of an SSD are actually quite different. The storage industry has only now started to design storage systems that take advantage of the differences in flash memory.

The Flash Translation Layer (FTL) translates the typical hard drive block-device commands and structure into comparable operations in flash memory. FTL is really a compromise for compatibility, since there is no need for the block and sector structure in flash. Additionally, the SSD controllers must perform a number of additional functions such as garbage collection, write amplification, wear leveling, and error correction, since the writeable life span of each storage cell of flash is limited (although there is discussion of a cure to this long-time flash illness). We’re going to see more applications that skip the need for FTL and take direct advantage of flash’s direct memory access capabilities.

High performance software capabilities such as databases currently circumvent the Operating System file system to attain optimal performance. Modern file systems such as Write Anywhere File Layout (WAFL), ZFS (which used to stand for the Zettabyte File System), and B-tree file system (Btrfs)are designed to take advantage of the various storage medias capabilities. The resulting systems were more efficient and easier to manage.

Storage system performance was a concern when operations were measured in milliseconds. It matters more on flash devices, whose operations are measured in microseconds. Future technologies like Memristor that will be even faster demand and optimized approach to long term storage and access of information. Compromises for convenience will exist but the penalties in performance will be high, impacting the application portfolio of organizations.

I mentioned that I was giving a presentation this week at the New Horizons Forum at the AIAA conference. Since it may provide some useful insight about the research underway at HP labs in a larger context, here is the content of one slide from that presentation:

1 datum is a point

2 data are a line

3 data are a trend

100 data are a picture

Having sensors to generate the data that fuels a more proactive business is important, but there is more to sensing than the sensors and the data collected. A holistic ecosystem view is needed. Unfortunately, this means that the tools of today may not be up to the tasks required.

You may have heard about HP’s efforts to place a million node sensor network in the ground for Shell, gathering seismic information. Traditionally, this kind of information was just a flash of perspective taken in the dark from a few locations. Instead, this sensing effort with Shell generated a much more fine-grained view, taken from a myriad of angles, to understand in-depth what was underground. 

In order to do implement the system, HP not only had to invent the sensors (relatively cheap and yet very sensitive MEMS devices), but we also create the networking and management techniques to make it useful. Building upon what we’ve learned, we’ve been researching whole new approaches to information storage and computation that will be required to generate value from massive amounts of information.

HP has many of the foundational patents on memristor devices and sensing techniques and we should soon see the shift in storage and computing that the implementation of these techniques should enable. The whole concept of computing will likely need to bow to the onslaught of information from sensing and the related metadata, changing how information is transferred within the computing environment -- shifting from computing on bits to analyzing information in graphs on highly parallelizes computing platforms: Cog Ex Machina

In addition, research is underway to understand how information can be analyzed, automated and displayed. New techniques can be applied to focus attention on the areas needing the creativity that people can provide.

In the marketplace, last year was the year of Big Data as a buzzword with its primary focus on generating insight from the massive amounts of information being collected. Frankly, that will not be enough for the future envisioned – we need to shift the focus to time-to-action, not insight and that is what many of our research efforts underway will enable.

Next week, I am going to be part of a panel at the New Horizons Forum that is part of the American Institute of Aeronautics and Astronautics (AIAA) conference. The panel is going to be focused on: Information Technology – Spin-On Technology for Aeronautics and Space. Essentially leveraging efforts into Aeronautics and space.

I’ll have a few minutes to present where I’ll focus on the shift in how we will use people and computing in the future and talk through some examples. In many organizations have so much data coming in that there is no way that any organization can effectively consume it, using today’s techniques.

I’ll use a simple model to show how things are changing and where HP labs is focusing research. The model is that one datum is a point, 2 data are a line and three are a trend. 100 data points are a picture or pattern. When you can move beyond calculations at the bit level and actions in isolation, whole new levels of capability open up.

Today, we think about working with data primarily in isolation. We need to start thinking about storing massive amounts of data (using devices like memristors), computing on the patterns using different techniques (like Cog Ex Machina) and using whole new approaches to focus the attention of those who need to act upon the context described by the data (like automation and gamification).

A holistic shift in how we approach sensing will be needed:

When I think about the work done with HP and Shell to gather detailed seismic information for oil detection and production, it makes me wonder about possibilities of families of sensors gathering data for planetary exploration. It also makes me wonder about using crowd sourcing techniques to tease out meaning from the unique items that pattern matching on all that data can identify.

It should be an interesting panel, for the audience and the panelists.

HP Labs researchers may have figured out a way to create a chip that generates neuron-like behavior, using a combination of memristors and capacitors to create a signal pattern similar to a neuron.

“To get the sort of spiking behavior seen in a neuron, the authors turned to a simplified model of neurons that allow them to transmit signals. When a neuron fires, sodium channels open, allowing ions to rush into a nerve cell, and changing the relative charges inside and outside its membrane. In response to these changes, potassium channels then open, allowing different ions out, and restoring the charge balance. That shuts the whole thing down, and allows various pumps to start restoring the initial ion balance.”

Each unit consists of a capacitor (to allow it to build up charge) in parallel to a memristor (which allows the charge to be released suddenly. The combination produces spikes of activity as soon as a given voltage threshold is exceeded. The researchers called this circuit a “neuristor.”

The illustration is courtesy of UC Berkeley.

A while back, I posted a likely capabilities comparison of a static memory solution based on Memristor technology to the current solutions based on Flash. Flash has always had an inherent reliability problem – you could only write a relatively few number of times before it would stop working properly. There was a great deal of work in the flash space to try and hide that problem from the users of the technology.

It looks like a Taiwan-based company Macronix may have found a workaround that reduces the flash memory fadeout problem. They put a layer of tiny ‘heaters’ in the chip, move the data out of the way every once in a while and cook that portion of the chip back to its native state.  They claim to allow for 100 Million Write cycles.

The team is going to present their findings next week at the: IEEE International Electron Devices Meeting (IEDM) in San Francisco. The technique doesn’t get around flashes relatively slow response and large size compared to other more advanced static memory approaches that are on the way, but it should allow the technology to be competitive longer (depending on how long it takes to move into production).

Their presentation is titled: Radically Extending the Cycling Endurance of Flash Memory (to > 100M Cycles) by Using Built-in Thermal Annealing to Self-heal the Stress-Induced Damage. The authors are H.-T. Lue, P.-Y. Du, C.-P. Chen, W.-C. Chen, C.-C. Hsieh, Y.-H. Hsiao, Y.-H. Shih, and C.-Y. Lu.