Perceptual Science Series

Ko Nishino (Columbia University)
The World in Eyes

 The World in Eyes

 Human eyes have been studied extensively in various research fields. Most of
such studies are concerned with the anatomy, physiology and visual
perception of the eye.  In this talk, we analyze the external appearance of
the eye.  It turns out that the combination of the cornea of an eye and a
camera viewing the eye form a catadioptric (mirror + lens) imaging system.
We analyze the characteristics of this corneal imaging system such as its
viewpoint locus, resolution and field of view.  Using this analysis we show
that we can compute a wide angle view of the scene surrounding the eye. In
addition, we can compute the projection of the scene onto the retina of the
eye.  In other words, from an image of his/her eye, we can obtain the visual
environment surrounding the person and what the person is looking at in this
environment.  We also show that from an image capturing both eyes of a
person, we can reconstruct the structure of the object the person is looking
at.  These results have direct implications in several fields.  In
particular, we show that the eye can be used to estimate the lighting
illuminating the person's face.  We demonstrate this in several face
relighting scenarios. We conclude by discussing implications of this work in
other fields such as human affect studies and HCI.  (Joint work with Shree

Contact RuCCS

Psychology Building Addition
152 Frelinghuysen Road
Piscataway, NJ 08854-8020


  • 848-445-1625
  • 848-445-6660
  • 848-445-0635


  • 732-445-6715