We consider the research and training activities of RuCCS members and associates to fall into three main thematic clusters: Visual Perception, Language, and Cognitive Theory and Foundations. These are briefly described below.

Graduate training in perception is an interdisciplinary effort designed to give students a solid background in (1) basic perceptual phenomena and their neurophysiological underpinnings, and (2) formal models of perception drawn from mathematics and computer science. Training emphasizes both empirical and theoretical issues, and this is accomplished through jointly-supervised research projects and cross-disciplinary course work. The perception community at Rutgers, drawn from the Departments of Psychology, Biomedical Engineering, Computer Science, as well as RuCCS and Julesz' Laboratory of Vision Research (which has now been combined with RuCCS), covers a wide variety of topics in both "early" and "high-level" perception and individuals' areas of expertise encompasses both empirical work and formal modeling. Faculty maintain active and visible research programs which have been supported by both federal grants and University funds. Faculty in perception work at maintaining close ties across academic disciplines and individual areas of expertise, and actively collaborate in supervision of student research.

One principal focus for collaborative efforts is the perception of motion, texture, color, shape and depth, selective attention and eye movements. Recent and ongoing interdisciplinary collaborations include several projects on nonlinear models of motion and texture perception and projects on models of localization of objects in space. Visual attention and the encoding of location and its cross-binding to motor control is another recent focus which bridges the interests of RuCCS researchers with those in the adjoining Laboratory of Vision Research (LVR). This research interest has also led to collaborative research with the CAIP Center (Computer Aids for Industrial Productivity), Computer Science, and the School of Communication Information and Library Studies (SCILS).

A second and growing area of interdisciplinary work is the development of formal models of object recognition and classification, with application to both human and machine perception. Key participants are two new faculty, Sven Dickinson (Computer Science/RuCCS) and Jacob Feldman (Psychology/RuCCS). Feldman's work, along with that of Michael Leyton, involves fundamental analysis of the genesis of form in perception. There is also an important collaborative effort centered around computer vision. This "Multimedia Imaging Group", directed by Casimir Kulikowski (Computer Science), which works on various aspects of medical imaging, automatic image segmentation and automatic semantics-based image retrieval. This group involves RuCCS and LVR researchers, as well as the Computer Science Research Laboratory and the CAIP Center. A number of research ties are maintained to faculty affiliated with CAIP, who focus on machine vision, human-computer interfaces design, speech recognition, and technical problems in virtual reality design, such as force-feedback.

Training centers around the jointly-supervised research described above and also relevant cross-disciplinary courses to give students a thorough grounding in both empirical issues and modeling. In addition, experience with applied problems in the design of computer vision and image analysis systems allows students to benefit from training opportunities provided by nearby industrial laboratories which carry out research in computer vision (e.g. NEC, SRI-Sarnoff Labs). Because vision is an integral part of the research focus of several Rutgers departments and labs, students find themselves taking courses in Psychology, Biomedical Engineering, Computer Science and Electrical Engineering, such as (in Bioengineering) Papathomas' course on Fundamentals of Vision, and Tzanakou's courses on Visual Pattern Recognition and Neural Networks, and (in Psychology), Leyton's course in Computational Vision, Pylyshyn's course on foundations of vision, as well as Dickinson's course in Computer Vision in the Computer Science department, which has attracted an interdisciplinary enrollment.

The language group works from strong interest in linguistic theory, coupled with experimental, formal, computational and theoretical research in language acquisition and processing. The research and training carried out by this group is informed by scholarship in linguistics, computational linguistics, psycholinguistics of sentence processing, neurolinguistics, learning theory, and the philosophy of language -- including foundational concerns about semantics and representation. Graduate students from all of the participating programs and departments who are interested in the cognitive science of language take advantage of the breadth and depth of the language group by taking classes and doing research with members of the language group.

A principal focus of theoretical linguistic research is on Optimality Theory, a new approach to grammatical theory developed by Alan Prince (Linguistics, RuCCS) and his collaborators Paul Smolensky (Johns Hopkins U.) and John McCarthy (U. Massachusetts, Amherst). The basic idea of Optimality Theory is that the grammatical calculation of well-formedness is accomplished through optimization with respect to a set of constraints on structure and on input-output disparity, rather than through the serial application of rules subject to filtering constraints. The constraints of Optimality Theory are held to be universal over all human languages, and conflicts between them are adjudicated by strict prioritization or ranking, rather than through any scheme of numerical weighting. Because Optimality Theory presents a grammatical architecture rather than an approach to a particular substantive subdomain, it has consequences for many areas of linguistic and psycholinguistic analysis. Prince's main area of research is phonology, particularly in its prosodic and prosodic-morphological aspects. Grimshaw has developed Optimality Theory in the context of syntax, focusing on theories of phrase structure and on the representation and acquisition of lexical information.

The second focus of theoretical linguistic research is also within a constraint-based framework, but one in which linguistic variation arises from small, parametric changes in constraints across languages. For example, through their research on the syntax of English, Romance and other languages, Viviane Deprez (Linguistics) and Ken Safir (Linguistics) seek to uncover the universal principles and the range of parametric variation necessary to account for the observed cross-linguistic phenomena. Research within the parametric and optimality-theoretic frameworks delineates strong constraints on the structure and representation of universal linguistic knowledge, enabling informed cross-disciplinary research in the processing and acquisition of language, as well as the cognitive lexicon. The Linguistics Department offers a wide range of graduate courses and seminars in syntax, semantics, morphology and phonology.

Research on human linguistic processing is strengthened by the multidisciplinary approach at RuCCS, in which evidence from many sources is brought to bear to determine the essential cognitive properties of linguistic abilities. The language group seeks to uncover the grammatical, computational, psychological and neural limitations on language comprehension that interact to constrain the architecture of the language processing system. Suzanne Stevenson (Computer Science, RuCCS) embeds a constraint-based grammatical theory within a novel constraint-based computational architecture, yielding a model of sentence processing that accounts for critical behaviors in human ambiguity resolution. The consequences of extending the model to Optimal Theoretic grammars are also being investigated (Stevenson, Grimshaw). Tesar pursues complementary research on the formal properties of Optimality Theoretic parsing algorithms. At a higher level, Jerry Fodor (Philosophy, RuCCS) has argued that certain formal properties are necessary for any computational architecture underlying human linguistic processing, providing further constraints on proposed models. Empirical studies provide not only the behavioral data which the group seeks to explain, but also yield crucial evidence about the processing architecture and its limitations, and the cognitive structures underlying this knowledge. Karin Stromswold (Psychology, RuCCS), Julia Boland (Psychology, RuCCS) and Fodor (Philosophy, RuCCS) contribute from this perspective by conducting psycholinguistic experiments that investigate the structure of lexical, syntactic, and semantic knowledge. Stromswold further explores the properties of the language processing architecture through the use of functional neuroimaging techniques (PET and MRI). Graduate courses and seminars in language processing are offered by the Computer Science Department (e.g., Natural language processing, taught by Stevenson), the Psychology Department (e.g., courses in developmental psycholinguistics and neurolinguistics taught by Stromswold and courses in psycholinguistics taught by Julie Boland).

Faculty in the Departments of Computer Science, Linguistics, Philosophy, Psychology and at RuCCS conduct formal, computational, theoretical, and empirical research that seeks to elucidate the constraints that enable the successful acquisition of language. Stromswold's empirical and theoretical research on the acquisition of language by normal and impaired children seeks 1) to delineate those aspects of language that are innate from those that must be learned; 2) to investigate the genetic and neural bases of language; and (3) to investigate parametric models of the acquisition and structure of syntactic representations in children and adults. Also working within a parametric framework, Deprez's empirical and theoretical research focuses on the structure and acquisition of Romance languages and English, and Safir's research focuses on the cross-linguistic acquisition of anaphora. Robert Matthews (Philosophy) focuses on formal learning-theoretic models of first language acquisition, especially on models based on parameterized linguistic theories. Working within the Optimality-Theoretic framework, Grimshaw is pursuing work on multiple sublexicons and the problem of apparent optionality and Tesar has designed a computational learning algorithm for Optimality Theory that solves aspects of the acquisition problem. Graduate courses are offered by the Linguistics Department (e.g., a special seminar on syntax acquisition taught by visitor R. Thornton), the Philosophy Department (e.g., a formal learnability seminar offered by Matthews), the Psychology Department (e.g., a language acquisition course by Stromswold), and RuCCS (e.g., seminars by our frequent visitors Jacques Mehler and Lila Gleitman).

A final focus of research interest in the language group at RuCCS is the connection between the structure of concepts and the structure of the semantic lexicon. These issues are connected by the assumption that whatever functions as the categories to which stimuli are assigned in perception, and as the constituents of thoughts in cognitive processes, should also serve as the semantic representatives of words in the speaker/hearer's internal representation of language. What psychological theory claims about concepts is thus constrained by what semantic (specifically linguistic and logical) theories say about word meanings; and vice versa. This convergence of concerns provides the possibility for a new integration of theoretical work in linguistics, philosophy and logic with experimental work in psychology. Among those whose work is directly involved with these issues are Fodor (structure of concepts; lexical semantics), Lepore (lexical semantics, theories of vagueness, supervaluation theory), Grimshaw (lexical decomposition and verb structure) and Leslie (conceptual structure in ontogeny and in autism), Schmidt (conceptual structure and planning), Leyton and Feldman (conceptual structure in vision and memory) and McLaughlin (connectionist approaches to conceptual and cognitive architecture). Under Fodor's supervision, a doctoral dissertation (in Philosophy) has just been completed on theories of concepts and their relation to problems of composition, and another (in Psychology) currently underway involves experiments testing linguistic theories about the lexical structure of English verbs. As a further example of the cross-disciplinary nature of these explorations, Grimshaw and Leslie have found remarkable similarities between the skeletal linguistic semantics needed for syntax and the primitive elements underlying early cognition. At a more abstract level, Prince and Feldman have discerned a number of formal connections between language and vision in the optimization of violable constraints, revealing the crucial role of constraint satisfaction in the processing of very different cognitive materials.

A third cluster, which cuts across the first two areas and also encompasses other theoretical fields in which RuCCS has particular strength, concerns such issues as the architecture of the cognitive system. This includes the nature of the underlying (computational) mechanisms of cognition, as well as the way in which these mechanisms encode information and the way that both the information and the mechanisms may change over time through learning or development. These are among basic theoretical and foundational questions on which RuCCS members have written extensively. For example, Fodor, Pylyshyn, Lepore, Stich, McLaughlin and Matthews have written on constraints on Cognitive Architecture, including its modularity and the requirements placed on it by the empirical facts of systematicity and productivity. These writers have argued, among other things, that the representational systems provided by the architecture must meet certain general but powerful constraints which exclude, for example, "prototypes" or analogs, or "pictorial" images, or any other noncompositional representational system, from being by itself adequate to the task. Areas in which RuCCS faculty have been leaders involve an analysis of the nature of concepts and their mental representation, the nature of mental images, and the fundamental problems faced by Connectionist models as models of cognitive processes (on which Fodor, Pylyshyn, McLaughlin, Stich, and Prince have written extensively).

Some RuCCS members or associates are concerned with the problem of representing knowledge in general (McCarty, Hirsh, Borgida, Amarel) or in particular domains, such as vision or language or planning and problem-solving (see the description of relevant research clusters). Some are concerned with understanding how representations change in response to relevant information -- i.e., learning in persons and machines (Hirsh, Amarel, Ellman, and Schmidt in the case of general knowledge; Fodor, Grimshaw, Matthews, Prince, Stevenson, Stromswold in the case of language; Julesz, Leslie, Dickinson, Feldman in the case of visual perception). Others are concerned to show how certain mechanisms are innate and/or universal or how they develop ontogenetically. For example, Leslie has shown that sophisticated cognitive mechanisms are present at very early stages of infancy (e.g. perception of causation and objecthood), whereas other mechanisms take a number of years to develop (e.g. ascribing mental states, such as beliefs and desires, to others).

The study of universals of language and perception also sheds light on the nature of the cognitive architecture. In this vein Grimshaw and Prince have argued that both phonological and syntactic differences among languages can be accounted for by the ordering which they place on a finite set of universal but violable constraints. Mark Baker's work on Linguistic Universals has direct bearing on the issue of cognitive architecture. Leyton, Feldman, Dickinson and Pylyshyn have shown that certain shape properties may be the (possibly innate and universal) basis for the encoding of shape in general. Stromswold has analyzed empirical evidence showing that certain structures are acquired despite impoverished evidence that lacks negative feedback.

The development of computational models of cognition raises both philosophical and methodological questions that many RuCCS members have addressed (e.g. Stich, Fodor, Pylyshyn). From the particular perspective of RuCCS researchers, the attempt to build specific computational models in domains such as language comprehension and visual perception goes hand in hand with designing an underlying architecture in terms of converging computational, linguistic, geometrical and psychological constraints. When this is done then the model's match to the observed human behavior is an inescapable consequence of that architecture, rather than based on mere mimicry. For example, Stevenson and Grimshaw are investigating the architectural requirements of a model of human parsing based on principles developed from Prince et al's Optimality Theory.

Two of the strongest groups on campus related to cognitive science are analytical philosophy (in Philosophy) and artificial intelligence (in Computer Science). We have already mentioned many of the philosophers since three of them are members of the Center and another 7 are active associates. They have been involved in the training of graduate students in analytical methods, in semantics, logic and the philosophy of language and mind, as well as in the foundations of cognitive science. Their participation constitutes one of our unique strengths, making us not only a center for critical theoretical and experimental studies, but also an institution whose science is informed by rigorous analysis of the foundations of the field.

The Computer Science (CS) Department is an active participant in the Center's activities. Four of our faculty members that also hold appointments in CS (Dickinson, Stevenson, DeCarlo and Stone). Our students have also benefited from the strong groups in artificial intelligence, particularly knowledge representation (with McCarty, Borgida, Kulikowski and Amarel) and machine learning (with Hirsh, Amarel, Ellman and Steinberg), as well as the Laboratory for Computer Science Research. The courses and research supervision that these members and associates offer, as well as their presence in our seminars, helps to strengthen the contact between the theoretical, computational and empirical specialties, which is at the heart of RuCCS' uniqueness. The connection with CS strengthens the idea of cognition as computation by encouraging students to explore various new methods from computer science that may provide ideas for architectures and processes that better correspond to what we know of human cognitive processing.

In addition to these links to computer science and to the study of the foundations of cognitive science there are also frequent collaborations between RuCCS members and individuals in other departments. For example, Robert Trivers (from the Anthropology Department) collaborates with Stephen Stich on evolutionary psychology. Eduardo Sontag from Mathematics works with members of the Discrete Mathematics center where there is an active interest in the analysis of neural networks and in algebraic models (which are central to the work of Schmidt and Leyton). There are even collaborations with the Graduate School of Professional and Applied Psychology (GSAPP) in which Leslie works with GSAPP's Autism School. GSAPP has also been involved in work on philosophical issues in psychopathology and psychotherapy, which has led to a course being given by Wilson (GSAPP) and Stich.

Applied cognitive science research also forms an important and growing part of our activities, especially through our growing association with the Computer Science Department and our collaboration and joint supervision of graduate students in the School of Communication, Information and Library Studies (SCILS). The Center is also home to the laboratory for Human-Computer Interaction as is a partner in the university's Digital Library initiative.

Because of our close proximity to a large number of universities and industrial research laboratories, we have close relations with laboratories such as NEC, Bell Labs and SRI-Sarnoff Labs, where some of our students have had summer internships. We have also had student exchanges with, for example, the Institute for Research in Cognitive Science at U. Pennsylvania. There have been regular visits among researchers at neighboring institutions. We have in the past had a "moving workshop" involving Princeton U., Columbia U., and NYU. These collaborations form part of our graduate training program.