In a previous post I wrote that at one time developers had to write a great deal of code that today would depend on prebuilt or commercial components.

There is one class of legacy program that represents the ultimate example of this - large apps that manipulate flat files for reporting purposes.  It is common in these instances to see extensive JCL devoted to sorting and copying, and COBOL programs doing substantial filtering, merging, and rewriting of data.  Understanding exactly what is happening is complicated by the use of multiple jobs and programs to paste this all together.  The fundamentals are actually fairly simple, once you work your way through the complex overlay of report processing that was written repeatedly.

I've seen programs that write out the table content, line by line, with a tag on the front indicating what type of report line it is. Then a final pass creates the actual formatted reports - kind of a primitive model, view, controller - with the model being the original input flat files, the controller being the multiple JCL and COBOL jobs filtering and massaging, and the view being that final COBOL program that emits 132 column lines suitable for your fan-fold, green bar paper.

Today, these programs would be built by linking a reporting tool to a relational database.  The endless JCL manipulations would be accomplished by predefined views or appropriate SELECTs.  And the results would be published to our corporate intranet with appropriate access controls.  Agility, in terms of our ability to produce new or modified reports, would be enormously enhanced and the labor costs associated with maintaining the reporting capability would drop by an order of magnitude.

This is a classic candidate for what Steve Woods has labeled asymmetrical transformation, which involves identifying those portions of your application that truly require custom coding and then ensuring that the rest are implemented by the appropriate COTS solution.

In my recent posts, Meet the Clones in My Neighborhood and You're Only as JUNG as You Feel, I mentioned how JUNG could be used to extract clusters of cloned code from a complex graph of an entire application. This has been very helpful to drill down into code to look at sub clusters as in the tiled image below. (I’ve substituted number for module names, since this is actual code.) But even with this simplified image, the team wanted to convey more meaning in our next generation tool: Visual Intelligence Explorer. The amazing thing about JUNG is that it allows our team to have complete control over the image rendering process. So we can control how the lines are drawn between the nodes. Up till now, with other tools, we’ve been coloring the lines, green for a weak relationship, red for a strong one. But using JUNG we’ve discovered lines convey meaning more intuitively when weighted instead of being colored.

This is especially useful in large graphs, where lines can hide other lines and get lost in the tangle. Line weights are natural and intuitive. Think of cable, rope, twine, string, and thread. Just the sound of those linear objects conveys strength. Cables pull barges, ropes rope cows, twine bales hay, string wraps packages, and threads? Well a thread is what I’m hanging by with this metaphor.

But just look at the image below, over 200,000 lines of code are represented by this graph of code modules, what’s important, what’s not? Let’s add some weights to those lines.

Here’s a zoom-in of the same image. Here we’ve made four types of lines; the weakest are dotted lines, the threads. Then we move all the way to our ropes and cables. The point is now we can see the most important cloned code.

Our Visual Intelligence Explorer is taking shape very quickly. Please check back here, soon I’ll be posting about using the text search engine Lucene to find legacy source code signatures.

I’ve been programming for so long, with so many languages, that sometimes I get a lofty feeling of having seen it all, done it all. What could possibly challenge me further? Look no further than Java generics and the JUNG (Java Universal Network Graph) library.

HP’s second generation of the Visual Intelligence Tools will use the Java Universal Network/Graph Framework (http://jung.sourceforge.net/). The 2.0 version released in April of 2009 has fully embraced Java generics, an improvement that came with Java 5.  Generics allow data structures in code to be more flexible and accommodating. And how could I complain when it’s such a powerful library. Just look at the balloon graph below: very cool. Our team sees several uses already, but first we must learn Java generics and moving the team to generics just doesn’t make us feel any younger. You see, the VI Tools uses network science to understand the relationship between modules of code. These bubbles could be a group of similar code. By allowing the nodes to arrange themselves; we get a better picture of what we like to call the unintended design.

Of course, my first move as team leader was to find the O’Reilly book that got the most stars on Amazon: Java Generics by Naftalin and Wadler.  Between each chapter, coffee is advised. But what I’m finding with generics is unbelievable power. I can see why JUNG went all-in with their design.

Once we get past Java generics, implementing JUNG in our Visual Intelligence Tools is our next hurdle. But once we’ve climbed these learning curves, we hope to have a better view of the tools that will end up on our next generation VI Tools.

What JUNG will allow HP Application Modernization services to do within the VI Tools is to programmatically stand among the nodes in the graph to provide a view that gives us a better sense of that these code duplication patterns mean. Standing within the graph allows us a greater ability to tie together the graph patterns and the code artifacts that produce them. More later from inside JUNG…

Hi. My name is Steve Woods and I work in HP Enterprise Services Application Modernization. I’ll be posting on topics that involve how we use some fairly innovative approaches to legacy transformation. In particular, I’ll discuss our Visual Intelligence Tools. Let me begin by covering the Woolsey and Swanson rule. In 1969, Woolsey and Swanson published a great book on operations research, Operations Research for Immediate Application: A Quick & Dirty Manual. Chapter Twelve: Final Instructions has the following warning to his students.

I recently showed this slide to the CIO of a large financial services company. He asked for a copy to frame and put on his desk. Seems every time he asks his staff to provide him with transformational opportunities this is the expression he gets. Legacy applications are around today for many reasons, but many organizations are living with the problem, not accepting the solution. I believe the toughest part of the “acceptance” is the acceptance of a transformational approach. Most accept that modern architectures work, but how do we get there? Can you even get there from here? In future posts, I’ll be addressing the Woolsey and Swanson impasse. I look forward to your comments.

I’d like to invite you to our virtual event; I’ll be discussing the Visual Intelligence Tools. Application Transformation: Getting to the bottom line.

For a quick preview check out Dana Gardner’s recent post:

http://blogs.zdnet.com/Gardner/?p=3300

Nov 5th, Americas: http://www.bitpipe.com/hpsolutionsconference/america

Nov 4th, Europe, Middle East, Africa: http://www.bitpipe.com/hpsolutionsconference/europe

Nov 3rd, Asia, Pacific and Japan: http://www.bitpipe.com/hpsolutionsconference/asia