ISÍ (I see): Bilingual language processing eye-tracking lab

Pennsylvania State University - ISÍ (I see): Bilingual language processing eye-tracking lab

Name of PI: Paola E. (Giuli) Dussias

Lab pictures and demonstrations
What I like to do when I take a break from having fun at work

Current Projects:
(1) Processing mixed language
Students working on this project: Rosa E. Guzzardo Tamargo; Jorge Valdés Kroff; Co-PI: Chip Gerfen.
Codeswitching performance has been analyzed primarily from the perspective of bilingual speakers; however, there are critical consequences for comprehension because unlike production, which is under the control of the speaker, comprehension is unpredictable. To investigate the relationship that exists between the frequency of occurrence of codeswitches in naturalistic data (production data) and the ease with which the comprehension mechanism processes these codeswitches, we have used eye-tracking methods to examine the reading of codeswitched sentences. Our findings show that sentences containing switches found in corpora (even if relatively infrequent) are easier to process than switches that do not occur in corpora. These results not only inform the debate in the monolingual literature about the relationship between comprehension and production mechanisms, but also enrich the theoretical foundation of codeswitching research by providing a measure of the predictive accuracy of grammatical and psycholinguistic models, and by suggesting critical and potentially neglected variables in the study of codeswitching.

(2) Processing morpho-syntactic information
2a. Grammatical gender processing during L2 and code-switched speech. Students working on this project: Nathan Hollister (Undergraduate PIRE Fellow); Lauren Perrotti (Undergraduate PIRE Fellow); Jorge Valdés Kroff (Graduate PIRE Fellow)
Although the study of reading comprehension can provide valuable insights into the mechanisms involved during language comprehension, we know that the principles that have been proposed to constrain written language processing are not always operant during spoken language comprehension. Hence, a better understanding of the cognitive processes involved during the comprehension of codeswitched language highlights the need to study spoken language comprehension as well. To achieve this goal, we have begun to employ an eye-tracking method known as the visual world paradigm, which has been widely used in psycholinguistic studies to investigate spoken language comprehension. In this method, participants see a visual display, consisting of line drawings representing different objects that are arranged within a grid. An instruction is provided through headphones, and the participant carries it out by clicking on one of the objects on the screen, using a computer mouse. Because eye-movements to the objects are time-locked to the unfolding speech signal, it is possible to study the comprehension of spoken language in real-time with a natural task. In collaboration with the graduate students in my lab, we have conducted a number of visual world experiments to determine whether grammatical gender is used by Spanish-English bilinguals to facilitate the comprehension of spoken codeswitched utterances.

2b. Processing Mood information in L2 Spanish
Students working on this project: Álvaro Villegas; Colleen Balukas; Amelia Dietrich; Jason Gullifer; Juliana Peters, Tim Poepsel; Hiram Smith. Although L2 speakers may be able to acquire syntactic knowledge in the L2 that is not present in the L1, processing differences between L1 and L2 speakers may still arise because L2 speakers may not be able to deploy syntactic information specific to the L2 during on-line L2 processing. In a series of studies examining the processing of mood and of control structures in L2 Spanish, we are beginning to examine these hypothesis.

(3) The role of verb bias information during L2 syntactic parsing
Student working on this project: Amelia Dietrich
Verb bias-or the tendency of a verb to appear with a certain type of complement-has been employed in psycholinguistic literature as a tool to test competing models of sentence processing. To date, the vast majority of sentence processing research involving verb bias has been conducted almost exclusively with monolingual speakers, and predominantly with monolingual English speakers, despite the fact that most of the world's population is bilingual. To test the generality of competing theories of sentence comprehension, we are currently conducting a number of studies using behavioral (reaction time measures and eye-tracking) and neurocognitive methods (ERPs) to add bilingual data to theories of sentence comprehension.


List of Members:

Current Graduate Students

Matthew Brown
Amelia Dietrich
Jason Gullifer (co-advised with Judith Kroll)
Pablo Requena
Álvaro Villegas
Lauren Perrotti


Current Undergraduate Students

James Graham. French; PIRE undergraduate student (Director, PIRE Project)
Nathaniel Hollister. Spanish; PIRE undergraduate student (Director, PIRE Project)
Lauren Perrotti. Spanish; PIRE undergraduate student (Director, PIRE Project)
Michelle Simon
Benjamin Stewart


Past Graduate Students

Geraldine Blattner. Department of French, Penn State University. Currently at Florida Atlantic University, Florida.
David Counselman. Department of Spanish, Penn State University. Currently at Ohio Wesleyan, Ohio.
Tracy Cramer Scaltz. Department of Spanish, Penn State University.
Rosa Guzzardo Tamargo. Department of Spanish, Penn State University. Currently at University of Puerto Rico, Rio Piedras.
Louise Neary. University of Illinois, Urbana-Champaign. Currently at Wesleyan College, Connecticut.
Amy Swanson. University of Illinois, Urbana-Champaign.
Jorge Valdés Kroff. Department of Spanish, Penn State University. Currently a Postdoctoral Fellow in the Department of Psychology, University of Pennsylvania.


Past Undergraduate Students:

Janna Bayliff. Musical Arts; Fall 2004.
Halle Belosh. Psychology; Fall 2005-Spring 2006.
Susannah Borysthen-Tkacz. Spanish and Linguistics; Fall 2008-Spring 2009 (Director, Honors Thesis). Currently at The Atrium School (MA).
Sofia Chernova. Psychology and Spanish; May 2009 (Director, Honors Thesis).
Timothy Deal. Psychology and Spanish; May 2010 (Director, Honors Thesis). Currently at Goddard Riverside Community Center.
David Denny. Psychology and Spanish; May 2009 (Director, Honors Thesis).
Elizabeth Francis. Pre-medicine and Spanish; May 2009 (Director, Honors Thesis). Currently at Penn State Milton S. Hershey Medical Center.
Jessica Goss. Spanish & Communication Sciences and Disorders; May 2010 (Director, Honors Thesis). Currently in Speech-Language Pathology at Temple University.
Lauren Perrotti. Spanish and Italian; Fall 2010-Spring 2012 (Director, Honors Thesis).
Gabby Pfeifer. Engineering and Spanish; Spring 2010.
Laura Quinn. Biology and Spanish; Spring 2010. Currently at Temple University School of Medicine.
Mackenzy Radolec. Biochemistry, Molecular Biology and Spanish; Fall 2011-Spring 2012 (Director, Honors Thesis).
Sarah Reed. Philosophy; May 2003 (Director, Honors Thesis). Currently at Deaf Hearing Community Centre, Swarthmore, Pennsylvania.
Christine Theberge. Communiation, Sciences and Disorders and Spanish; Fall 09-Spring 2012 (Director, Honors Thesis).
Celina Troutman. Computer Science and Bachelor in Philosophy; May 2003 (Director, Honors Thesis).


Current Collaborators

Dr. Teresa Bajo, Universidad de Granada, Spain
Dr. Josep Demestre, Universitat Rovira i Virgili, Spain
Dr. Chip Gerfen, Penn State University
Dr. Carrie Jackson, Penn State University
Dr. Judith Kroll, Penn State University
Dr. Maya Misra, Penn State University
Dr. Jill Morford, U. of New Mexico
Dr. Pilar Piñar, Gallaudet University
Dr. Rosa Sanchez-Casas, Universitat Rovira i Virgili, Spain
Dr. Janet van Hell, Penn State University.