Applications Services Blog
Get the latest thought leadership and information about the role of Applications Services in an increasingly interconnected world at the HP Blog Hub.

Software Development Methodology - The Search for Ignorance Reduction

By: Leroy Mason, A Global Chief Technologist, Hewlett Packard Company

 

head in sand - condensed.jpgThere’s a view that software is not a product in itself but rather a medium for storing knowledge. In this view, working software is more like DNA, brains, and books than it is a box of Cheerios. Software is the focus in enterprise applications because the stored knowledge becomes directly active on the problem to be solved. The other forms of stored knowledge enables an intermediary, e.g. a person, to act being guided by the knowledge.

 

This view has implications for how we produce software. If we are not churning out a product, then speaking about “cookie cutter” and “industrialized” approaches is misleading, however popular. No, if we view software as stored knowledge then this means software development is the process of building knowledge. Expressed in a negative way, software development is about reducing ignorance.

 

How useful is this perspective; that software development is best thought of as knowledge acquisition? Well let’s conduct a simple thought experiment. Imagine we are trying to develop an enterprise application with no methodology or discipline at all. This means there is no attempt to specify requirements or write a design first or conducting testing – we just start coding (…and don’t even think about configuration management!). As the enterprise application is built and executed by users there comes a point where the validity of the code is assessed. This identifies what is in the code that is correct and incorrect. Users inform the developers and what is incorrect is transformed to correct.  This process continues over a possibly infinite duration (remember this is a thought experiment).

 

This process is simply the mechanism that is being used to both increase the knowledge in the code and remove the lack of knowledge, or put another way to reduce the ignorance in the code.

 

At the end of this project we are left with mostly knowledgeable code. This system can then work to the satisfaction of its users. However if the process is cut short there may still be a small amount of ignorance left in the code that represents the rest of the journey to find the knowledge. Also, we are only interested in retaining the knowledge – we do not make any effort to retain the bad code – and this is not retained in any way. The knowledge in this code can be quickly invalidated if new knowledge arises which will require backtracking to rework the system. In this process the final software is contaminated with the process used to build it. The code is a write-only knowledge store. It is impossible to read it. In this approach we use the activity of building the system to acquire knowledge of the system. The code is the product, which in turn is the knowledge.

 

As this thought experiment shows, if the knowledge collected in the process of building the software is fixed only in the code, then we are unable to show, read, update or otherwise use that knowledge. Simply building knowledge into the code is not the best we can do. For everything we know, we also have a certain amount of residual ignorance retained in the code. 

 

Ignorance is the other side of the knowledge coin. If system development is the acquisition of knowledge then it is also correct to think of it as the reduction of ignorance. Let’s take a look at what kinds of ignorance there might be.

 

Philip G Armour describes “five orders of ignorance” that are helpful in understanding what is needed to reduce ignorance in order to economically write software that works.

 

0th Order Ignorance (the "zeroth" order) is no ignorance at all. It is when I know something and can demonstrate it. For example, for an enterprise application in production, I can reliably estimate the effort required to make a small change to an existing form.

 

1st Order Ignorance is when I don’t know something but I am able to readily indentify that fact. For example, I am asked to add a comment field to an enterprise application form but I do not know if plain text will suffice or it needs to be rich text. To reduce 1st order ignorance to 0th order ignorance, I need to ask the business what they want.

 

2nd Order Ignorance is when I don’t know that I don’t know something. That is, not only am I ignorant of something I proceed as if this does not matter. For example, a developer is asked to add workflow to an enterprise application but due to their limited experience is unaware that good third-party workflow COTS solutions exist as alternatives to bespoke development.

 

3rd Order Ignorance is when I don’t have a suitably efficient way of finding out that I don’t know something. This is the lack of a process and it is a major problem. The “suitably efficient” constraint needs to be observed with regard to 3OI because there is always an inefficient way to find out what we don’t know – just build the system (as in the thought experiment above) and the customer will tell us what we didn’t know and ask for it to be fixed. If we have a suitable software development methodology then we do not have 3OI - if we use it properly. In this view a software development methodology does not provide answers – instead it identifies and clarifies questions (of 1st order of ignorance) that can simply be answered.

 

4th Order Ignorance is when I don’t know about the five orders of ignorance. Having read this far you are cured of this.

 

The process of efficiently developing software can be cast as a process of reducing higher orders of ignorance to lower orders and eventually ending up at 0OI.

 

The challenge for enterprise applications professionals is to navigate from ignorance to knowledge while keeping the customer engaged. If we are up front in talking about what we don't know, or what the customer doesn't know, then the engagement is likely to be fairly short. Clear demonstrations of crisply held knowledge will help the customer to build confidence in a service provider. However if these are too convincing then we risk losing sight of the need to reduce ignorance.  For example, if you present a prototype that works really well, the customer will think you're almost done - so don't do this. Even if the prototype is pretty slick at least add a "crash" button - something innocuous but when pressed brings up an awful error message. When the customer asks what happended, explain that it represents what we don't know yet.

  

References

  • Armour P. G. “The Five Orders of Ignorance”, Communications of the ACM, October 2000, Vol 43 No 10

 

About the author

 

Leroy Mason.jpgLeroy Mason, A Global Chief Technologist, Hewlett Packard Company

Leroy is global chief technologist of Microsoft Application Innovation Services for HP. He has 30 years of experience in large-scale enterprise application services including Wall Street banks, Microsoft, and HP. His professional interests include adapting the applications lifespan economics to cloud platforms.

 

Comments
tarquinhall(anon) | ‎07-05-2014 11:45 AM

Really a great post. Keep sharing like this.

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the community guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
Showing results for 
Search instead for 
Do you mean 
About the Author
. . Service Systems . .


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation