Perceptual Science Series

Anthony Hoogs (GE Global Research, Niskayana, New York)
Video Content Annotation Using Visual Analysis and a Large Semantic

September 29, 2003 at 2:00 p.m.

Psychology Room 101, Busch Campus

Anthony Hoogs

GE Global Research

Video Content Annotation Using Visual Analysis and a Large Semantic Knowledgebase

We present a novel approach to automatically annotating broadcast
video.  To manage the enormous variety of objects, events and scenes
in video problem domains such as news video, we couple generic image
analysis with a semantic database, WordNet, containing huge amounts of
real-world information.  Object and event recognition are performed by
searching WordNet for concepts jointly supported by image evidence and
topic context derived from the video transcript.  No object- or
event-specific training is required, and only a few object models and
detection algorithms are required to label much of the significant
content of news video.  The hierarchical structure of WordNet yields
hierarchical recognition, dynamically tailored to the level of
supporting image evidence.  The potential of the approach is
demonstrated by analyzing a wide variety of scenes in news video.

Joint work with Jens Rittscher, Gees Stein, and John Schmiederer



Contact RuCCS

Psychology Building Addition
152 Frelinghuysen Road
Piscataway, NJ 08854-8020


  • 848-445-1625
  • 848-445-6660
  • 848-445-0635


  • 732-445-6715