Does structured vs. unstructured really matter anymore?
I was in a discussion the other day about the state of automation and IT. We started talking using a quadrant chart (much favored by consulting organizations today), that had an axis for data and one for process that looked something like this.
As we began to talk using this model, it became clear that the terminology is not nearly as meaningful as it seems. Most of the IT efforts relating to big data and automation are at their foundation focused on imposing structure on the unstructured.
Whether it is sentiment analysis or context recognition, a great deal of work is being spent recognizing patterns and shifting to work on the patterns (as the new structure) rather than the data itself. Granted, we do have to store the unstructured data now so that later when structure can be derived, we can gain greater value.
A similar approach can be applied about process. As we determine the structure we know how to deal with, it becomes normal and we can automate it.
We are constantly looking for a threshold when dealing with the unstructured and the point when we cross it – since at that point it becomes structured. Another big issue about where we look for thresholds is how often that event happens. If it is very rare, it may not be worth the effort. Determining which areas to “structure” may be one of the key skills differentiating IT innovators in the coming years.
All the discussion about “the long tail” and its value is great, but the opportunity for big organizations is to pull in the long tail using their technological clout, so it doesn’t get stepped on by the newcomers, who can use that unaddressed market segment to their advantage.
One thing to keep in mind is: sometimes those unique events are where the value is. In an attempt to place structure, we need to be sure not to act like a lemming and jump off the cliff because that is what we’ve always done. Important information can be lost on this road to structure.
