Is the Internet Robust?

[#1]

The Internet, along with Web content and applications, is replacing alternative forms of communication and access to information and it is changing the way the world works.  Market forces will drive elimination of diversity of web technology putting these capabilities at risk of pandemic failure.  A pervasive failure could paralyze advanced civilization.

I work from home.  If I lose Internet access, in the short term, I am severely handicapped, and very soon I cannot to do my job.  My alternative is to drive to another location where I can get Internet access and work from there, but what if Internet communications or most web servers weren’t working.

Email is a primary means of communication.  The internet is replacing traditional telephone communications. Web pages are a primary source of general information or shared documents as well as web-based teleconferences.  Internet failure or widespread failure of computers on the Internet would have devastating business effects.  The web is a significant part of retail sales.  The Internet is an integral part of transportation management and supply chains.  Retailers also depend on Internet access for point of sale operations.  Healthcare will increasingly depend on Internet communications for access to health records and management of treatment. There’s no going back. As we become more sophisticated and our systems more optimized, we increase the risk to our way of life.  In “Our Increasingly Fragile Civilization,” I talked about a lost, ancient civilization and the risk of Internet failure.

The Internet is considered robust because of redundancy.  If some servers or communication links fail, the traffic is re-routed through alternative paths.  However, economies of scale promote the elimination of diversity in network facilities and the computers connected to the Internet.  This homogeneity makes the entire web vulnerable, just as the popularity of the Windows operating system makes it a prime target of intrusion.  The more homogeneous the Web, the more damage can be done by a single virus.

Contemplation of this risk led me to look on the Web for sources of insight on system robustness.  Some authors focus on redundancy, which is the first line of defense against failure.  But redundancy is of no value against a pervasive failure.  A second line of defense discussed by some authors is diversity.  If components with similar functionality implement diverse solutions then it is less likely that a pervasive event could cause all of them to fail. 

However, I found an excellent essay by Gerald Jay Sussman, “Building Robust Systems.”  Sussman develops insight from analogy to biological systems.  He describes a robust system as having redundancy, componentization, degeneracy, exploratory behavior, generality and regeneration [my summary].  A robust system should not only be able to survive failures, but it should be able to adapt and evolve in response to changing opportunities and adversity.

Redundancy we understand.  Componentization partitions the system into relatively independent components such that failure tends to be contained within a component.  Generality means designing components to function in circumstances not anticipated by the designer; this is fairly common practice.  Degeneracy involves the availability of alternative capabilities that solve the same or similar problems; this tends to be eliminated in favor of optimization.  Degeneracy might be viewed as redundancy with diversity.  Exploratory behavior involves generating and testing alternative solutions so that the system can adapt to change; this may be avoided as duplication of effort.  An exploratory system might also generate new solutions to replace existing ones.  Regeneration involves action to repair or replace damage to the system—self-healing.

Sussman goes on to discuss how these concepts can be applied to software design.  I recommend reading the essay.  My brief summary obviously just scratches the surface.