HP Labs develops multimedia content technologies for cloud and mobile applications
Contributed by Simon Firth, freelance technology journalist
Intelligent analysis of images and videos is finding its way into ever more applications – from security and surveillance, to entertainment and smart phones. It can be used to find and recognize faces, clothing, road signs, and other objects; and to adapt multimedia content for display in all kinds of mobile environments.
“The key challenge is to be able to quickly analyze and adapt this content for viewing, search, indexing, and organization,” says Qian Lin, of HP’s Printing & Content Delivery Lab.
Lin heads a team at HP Labs focused on meeting this challenge via a set of multimedia content technologies that run as cloud services. They intelligently analyze multimedia content for tagging and summarization, and adaptively format it for mobile screen consumption. In addition to adapting content for small displays, the technologies also make it easy to compose print publications.
A better scanner
HP Mobile Document Capture, for example, uses the camera on a mobile phone to photograph items like letters, memos, receipts or contracts and then turns them into ‘living’ digital documents.
Many mobile scanning apps create pictures of documents that can be stored in a digital database, but the HP application is unique in its ability to then recreate the document with the correct aspect ratio and image/text separation, making it readable by OCR software.
As a result, a captured receipt, for example, can be automatically entered into a budgeting spreadsheet, or a section of a printed memo can be quoted, or edited, in an email.
“For cloud providers, this offers an onramp to creating cloud-based document repositories from which you can build workflows of any kind,” says Lin.
Spotting faces, even when they’re moving
Faces have also been a long term research interest for Lin’s team. Their industry-leading face recognition software has already found its way into HP products such as the HP Snapfish photo service.
More recently, the technology has been incorporated into HP-owned Autonomy’s search products. That’s thrown up some interesting new challenges, Lin reports.
“A lot of the material that Autonomy wants to scan is video,” she says. “It’s also often shot in low light or with the figures far in the distance and not facing the camera. So there’s a lot for us to work on.”
Making multimedia content technologies available on HP Cloud
“We’re now offering the Mobile Document Capture as an API, along with several other APIs that people can use to build cloud service applications,” reports Lin.
The idea, she says, is that people can create web and mobile apps with this advanced functionality without having to develop these technologies on their own.
The software that enabled both APIs is now available at HP Cloud as part of a broader HP Labs effort to create useful data points across the entire media content ecosystem.
“This is very much about deeply understanding the content in multimedia, a lot of which is unstructured, and asking what are the things that people are interested in?” says Lin. “If we understand it, we can tag it and then we can add value to things like searching or sharing, and multimedia consumption or production – and we’re actively looking into those and many other areas.”
