Mentor: Jacob Feldman
Project Title: A world of autonomous agents.
Feldman and graduate students have recently created a programmable artificial life environment containing a population of autonomous virtual agents. The platform has two complementary goals: a) to create model-driven autonomous agents capable of intelligent behavior, and b) to investigate human interpretation of that behavior, focusing on a timely topic in cognition, namely, the perception of intentionality. Each agent is an autonomous program capable of a variety of behaviors, such as foraging for food, interacting with other agents, and even competing for survival. The agents are endowed by their programs with goals, intentions, perceptual capacities, the capacity to explore and learn in their environment, and the capacity to plan their future behavior. Programming new agents is a creative but feasible exercise for novices because the agents' programs are written in a modular language built on basic behavioral and perceptual building blocks. Undergraduate researchers can endow them with new perceptual faculties, improved action planning, or even new goals and competitive frameworks. They can also assess how well and under what conditions human observers are able to interpret the agents' mental states, that is, deduce what the agent was "thinking" based on its actions. This virtual environment thus serves as a novel platform for investigating key problems at the interface of computer science and perceptual psychology, including: visual interpretation of complex dynamic displays, population dynamics of artificial perceptual agents, and computational procedures for interpretation of intentional action. Students with a programming background can participate in the development of new agents and interactive interfaces.
Mentors: Jacob Feldman, Manish Singh
Project Title: Part-based studies of shape.
Shapes are often represented according to their part structure: examples include the "skeletal" model developed by Singh and Feldman. Such models can be used to recognize objects and create abstract imagery by simplifying and removing parts, and computation of relevant geometric features such as local symmetries. Projects can refine these models by representing new kinds of part organization (e.g., operations that create texture) or encoding constraints on part structure for specific classes of shape (biological forms or man-made artifacts or moving shapes).
Mentor: Randy Gallistel
Project Title: Automated Cognitive
NeuroGenetic Screening Using a Custom Matlab Toolbox
Run a platoon of genetically manipulated mice through a computer-controlled sequence of tasks in a fully automated 24/7 live-in test environment and analyze the data using a Matlab Toolbox developed for analyzing the kind of time-stamped event record obtained in these experiments. Project involves learning to use the toolbox and, in the process, helping to improve the manual and tutorial materials that we are preparing to enable other labs to use this system. The tests are designed to detect heritable malfunctions in basic mechanisms of cognition, particularly those mechanisms that enable animals to localize themselves in space and time.
Requirements: Course work in statistics; Matlab programming skills.
Mentor: Eileen Kowler
Project title: Visual search
Visual search is an operation that is performed by humans and by machines (for example: robots navigating through cluttered environments, or people searching through a desk for a set of keys). This project will require students to develop a simple but realistic visual search task for humans to perform (for example, finding an object hidden under another on the screen) where the likelihood of finding the target in a particular location is traded-off against the cost (search time or effort) of accessing the most likely locations. Performance will be measured by recording movements of eye or arm. The search pattern adopted by the human participants will be compared to models of optimal strategies. For further information about lab activities, see: Prof Kowler's web page.
Requirements: Matlab programming skills. Coursework in statistics recommended.
Mentor: Smaranda Muresan
Project title: Computational Models of Language Learning and Understanding
Humans are uniquily equipped to talk about their perceptions and actions, by producing and understanding sentences that have not been uttered before. Prof Muresan has developed a computational framework for learning syntactic-semantic grammars able to both understand and generate natural langauge expressions. The learner is exposed only to a limitted number of positive examples consisting of utterances paired with their syntactic-semantic representations. While interleaving syntax and semantics has proven successful in eliminating a certain degree of ambiguity inherent in human language, the system needs to be extended with a probabilistic modeling component --- language expressions are more or less likely to appear in a certain context. The probabilistic component is currently being implemented as a prototype in the system. The summer project aims to bring together students from computer science and linguistics working togther on two main aspects of this framework: 1) implementing the prototype of the probabilistic component in Java/C++ or other efficient programming languge and integrate it in the current framework; and 2) extend the training set of the learner by annotating additional data. The first aspect will require background in computer science, while the second will require background in linguistics. For annotation, the student will have access to an annotation tool developed for this framework. The training data developed will be used in the learning framework enhanced with the probabilistic component.
The students will be able to participate in other research activities, such as reading groups and research meetings of the Laboratory for the Applied Language Technology and Society at the School of Communciation and Information.
Mentor: Mor Naaman
Project title: Big Data Temporal Analysis of Twitter
This Big Data project will focus on the analysis of temporal patterns in large scale data from Twitter. The research will involve developing a data analysis framework using Hadoop/MapReduce to analyze hundred of millions of Twitter messages collected since September 2009, and expanding the data collection mechanism to include new data sources. The student will write Hadoop MapReduce scripts to analyze large scale data, and help build a wrapper or Web-based interface around those scripts to enable non-programmers to query and visualize the data. The student will also help extend a crawling mechanism to collect data from Twitter and other social media sources.
Requirements: Interested students should have experience with data mining and web programming, as well as a deep interest in social aspects of computing and an inherent ability to have fun.
Mentor: Andy Nealen
Project title: Shape Modeling and Animation
Computer software for creating, modifying and animating digital 3D shapes is a key component in computer aided design, engineering, computer animation and digital games. The student will be involved in research into dynamic hierarchical part hierarchies, which combine the interrelated topics of modeling and animating digital shapes into a coherent whole, as well as and simple and effective sketch-based computer tools, with the ultimate goal of making these tasks more accessible to amateurs and professionals alike. The research will build upon previous work on systems such as Fibermesh, but put an emphasis on developing new sketch-based interfaces, algorithms and shape representations that allow us to combine the previously mostly disjoint, yet intrinsically connected tasks of shape modeling and animation.
Requirements: basic programming (in C/C++) and computer graphics (CS 428 equivalent) would be very beneficial.
Mentor: Andy Nealen
Project title: Shape, Color, Motion and Input in User-Centric Video Game Design
Video games have established themselves as an everyday cultural artifact. As a result, they have become a prominent, successful and expressive form. This research will explore how humans perceive a video game environment, and how this environment influences common control tasks. Check out the silencing illusion for a dramatic illustration of why perceptual constraints matter in dynamic game displays.
While early day designers of computer games were constrained by technical limitations, we believe that abstraction as a game design paradigm is an important information visualization tool, and that artificial constraints on video game rendering serve the goal of making the game more accessible. The research objectives are (a) to provide algorithms for the creation of an entire spectrum of pixel-, vector- and line-based visual abstractions from photographs and 3D models, and (b) to prove, by user study, that non-representational video games, combined with just the right kind (and complexity) of user control, are preferred by the majority of human users.
Requirements: basic programming (in C/C++) and computer graphics (CS 428 equivalent) would be very beneficial.
Mentor: Thomas Papathomas
Project title: The role of schema driven and data-driven signals in 3-D visual perception.
One theory on how we we perceive three-dimensional (3-D) objects is that the data driven signals (motion parallax, binocular disparity, occlusion, etc.) are "interpreted" by schema-driven processes to arrive at a final percept. Thus, in this view, the input from our sensory systems is influences by prior experiences, expectations, suggestions, and other cognitive processes. We experiment with physical and virtual 3-D volumetric stimuli, such as faces and scenes, to study the roles of data driven and schema-driven processes.
Requirements: Background in experimental methods. No lab experience is necessary; coursework exposure is adequate.
Mentor: Vladimir Pavlovic
Project title: Social analysis from videos
The goal of this project is to understand interactions among people through computational analysis of live and recorded video footage. Our setting is that of a classroom: we seek to understand who is in the classroom, when and how students interact with each other, and how they interact with the instructor. To do this we have to recognize subjects in video, track their movement, understand their actions, and infer interactions. We are looking for an undergraduate researcher who will help with data collection and processing (video recording, annotation) and implementation of basic video recognition and tracking algorithms, such as face and object recognition.
Requirements: The student should be familiar with programming in C or C++ and have knowledge of (or be willing to learn) OpenCV, an open source computer vision library. Knowledge of linear algebra is a plus.
For more information see this project page.
Mentor: Anselm Spoerri
Project title: The expressive possibilities of perceptual technology.
For the last two years, Spoerri has collaborated with undergraduate students in his Information Visualization class to develop together whereRU, a digital media portal that enables users to experience Rutgers in a visually immersive way. The underlying technology, based on recent advances in computational photography (high-resolution panoramas --- Gigapans --- and image kaleidoscopes --- Photosynths), has a flexible API, allowing a computer science undergraduate to develop extended capabilities for navigation, interaction and display. Research opportunities extend to scientific communication itself – an information science undergraduate might make a project of immersively documenting the REU site itself using these emerging technologies.
Mentor: Matthew Stone
Project title: Modeling interactions between humans and dialogue systems
Start from an existing data set of interactions between human subjects and a dialogue system, including text, context and interpretation. Apply existing machine learning tools such as maximum entropy methods to build models of this data, such as: what dialogue moves speakers tend to make, how speakers tend to express those moves in words, or what outcomes different dialogue moves tend to achieve. Focus on the creation of features, guided by results on conversation from cognitive science, that capture the natural coherence of dialogue. Evaluate the new models offline using techniques such as cross-validation that assess how well they characterize the existing data.
Requirements: undergraduate coursework in AI, data mining or computational statistics; experience with cognitive psychology helpful but not essential.
Mentor: Matthew Stone
Project title: Building computational dialogue systems
Carry out human-subjects experiments to contrast the performance of two different versions of a computational dialogue system that identifies objects with human users. One version will be a baseline system; the other will be an improved version (perhaps built in collaboration with a linguistics, psychology or AI student) with additional expressive capabilities or optimized decision-making algorithms. Develop hypotheses about qualitative and quantitative performance differences between the systems, come up with conditions and metrics to measure the differences and assess whether the hypotheses are confirmed using appropriate statistical tests.
Requirements: undergraduate coursework in user interface design or experimental methods (and ideally both).
Mentor: Matthew Stone
Project title: The grammar of complex referential descriptions
Elicit textual descriptions of complex objects in a visual scene from informants. Characterize the linguistic constructions and discourse strategies used, focusing on a specific dimension of descriptions such as the interpretation of visual-spatial words (color, shape, or arrangement), reference to the part-whole structure of objects, or the choice of distinctive properties or relationships of objects. Formalize the linguistic resources involved in a computational grammar formalism and create a demonstration reproducing attested descriptions using existing tools for grammar-based parsing and generation.
Requirements: undergraduate coursework in linguistic syntax and semantics; experience with programming helpful but not essential.
Mentor: Matthew Stone
Project title: Strategies for collaborative reference
Elicit problematic dialogues where two subjects negotiate to identify objects in a visual scene. These dialogues can be created by asking subjects to distinguish between unfamiliar, similar or complex objects. Build a descriptive model of choices that subjects make in resolving problems in dialogue, focusing on speakers' uses of specific dialogue moves such as hedges, reformulations or clarifications. This involves studying models of negotiation from cognitive science to create hypotheses about the dynamics of specific discourse moves, developing qualitative and quantitative measures that could distinguish among the hypotheses, finding and annotating the target discourse moves as part of a team or crowdsourced effort, and using the data to test the hypotheses.
Requirements: undergraduate coursework in empirical study of human behavior, ideally but not necessarily through coursework on psycholinguisitics, communication or discourse.
Mentor: Elizabeth Torres
Project: Models of human movement
Build an interface that captures positions and orientations of real time hand movements (using available sensor data) and outputs "fake" feedback in real time. The underlying algorithm should fool a human performer into believing they are moving in a certain way, which in reality is different from the actual way in which they are moving. Trial to trial repeats should be stochastic enough that reveal no possible pattern, yet in the long run some pattern should emerge to drive the motor and perceptual systems to adopt new performance patterns.
Requirements: Knowledge of Matlab and C++ is recommended.
Mentor: Elizabeth Torres and Thomas Papathomas
Project title: Vision for action versus vision for perception.
Research on the characteristics of the ventral neural system (also referred as the "what" system, or the "vision for perception" system) and the dorsal neural system (also referred as the "where" system, or the "vision for action" system) is one of the long-standing debates in vision research. This project combines two main tools to test the hypothesis that action is governed by the dorsal system, even in the presence of conflicting input from the ventral system. One tool is three-dimensional bistable stimuli that provide conflicting inputs; the other is an elaborate computer-monitored system that can detect and measure motion by body parts in real time.