How UIA Completes the Big Data Picture
|Copyright 2014 by Virgo Publishing.|
|Posted on: 10/31/2012|
By Sid Probstein
Many people say the Big Data movement is about "unstructured data," but they are missing an important point. Log files and click streams are not really unstructured; they are just relatively unfamiliar, and sometimes variable structures. What about the other sources that contain important information about customers and buying habits? These offer a wealth of value that is today largely untapped. Emails, open-ended survey questions, Web forms, call logs, discussion boards, SharePoint and Wiki sites — this is the true "unstructured content" that completes the picture of customer perception. It is moreover the best source to create a useful internal view — employee and partner behavior for example.
Unstructured data is not that different from structured data. It tells you what happened, and probably where. Unstructured content, on the other hand, explains WHY things happen. The inability to process and analyze this unstructured content is what prevents most of the Big Data players from presenting a comprehensive view. Aggregating and analyzing unstructured content is challenging for the following reasons:
Vendors in the unified information access (UIA) space have been focused on aggregating, enriching and analyzing unstructured content — as well as data — for years. These vendors provide technology that complements Big Data infrastructures by bringing unstructured content into the analysis framework and by presenting Big Data in context to remove information blind spots in business applications and automated business processes.
Recent IT market consolidation renders the Big Data arena an even more difficult space to compete in. Partnering with a UIA expert gives companies an advantage over the big stack mega vendors. A $3.5 billion market in software and services, UIA is one of the fastest areas of IT development, growing at 24 percent CAGR according to IDC. UIA technology includes the essential text analytic capabilities such as entity, concept, key phrase and sentiment analysis that help transform unstructured content into meaningful insight. This provides the critical "why" that allows companies to act swiftly and decisively, gaining a competitive edge with significantly faster time-to-value.
Sid Probstein is co-founder and CTO of Attivio, responsible for product and technology strategy, implementation and delivery. He has more than 20 years of experience in managing R&D organizations and delivering award-winning, high-value enterprise software and solutions.