You can watch a somewhat dated film (in German) about our work here. Two other films describe ongoing research in our department: see here (English) ; and here (German). The Vasishth Lab Blog is here.
Current active PhD students: Felix Engelmann, Anna Laurinavichyute, Bruno Nicenboim, Dario Paape, Farnoosh Safavi.
Alumni: Sven Brüssow; Katja Suckow; Kate McCurdy; Dr. Titus von der Malsburg; Dr. Umesh Patil, Dr. Samar Husain (postdoc 2011-2014), Dr. Pavel Logačev,Dr. Lena Jäger,Dr. Paul Metzner.
Postdoctoral researcher: Dr. Lena Jäger.
Current active collaborations: Frank Burchert, Stefan Frank, Sandra Hanne, Rick Lewis, Frank Rösler, Narayanan Srinivasan, Julie Van Dyke, Samar Husain.
An English double-embedded relative clause from which the middle verb is omitted can often be processed more easily than its grammatical counterpart, a phenomenon known as the grammaticality illusion. This effect has been found to be reversed in German, suggesting that the illusion is language specific rather than a consequence of universal working memory constraints. We present results from three self-paced reading experiments which show that Dutch native speakers also do not show the grammaticality illusion in Dutch, whereas both German and Dutch native speakers do show the illusion when reading English sentences. These findings provide evidence against working memory constraints as an explanation for the observed effect in English. We propose an alternative account based on the statistical patterns of the languages involved. In support of this alternative, a single recurrent neural network model that is trained on both Dutch and English sentences indeed predicts the cross-linguistic difference in grammaticality effect.
Comprehension of non-canonical sentences can be difficult for individuals with aphasia (IWA). It is still unclear to which extent morphological cues like case-marking or verb inflection may influence IWA’s performance or even help to override deficits in sentence comprehension. Until now, studies have mainly used offline methods to draw inferences about syntactic deficits and, so far, only a few studies have looked at online syntactic processing in aphasia. We investigated sentence processing in German IWA by combining an offline (sentence-picture matching) and online (eye-tracking in the visual-world paradigm) method. Our goal was to determine whether IWA are capable of using inflectional morphology (number-agreement markers on verbs and case markers in noun phrases) as a cue to sentence interpretation. We report results of two visual-world experiments using German reversible SVO and OVS sentences. In each study, there were eight IWA and 20 age-matched controls. Experiment 1 targeted the role of case-morphology, while Experiment 2 looked at processing of number-agreement cues at the verb in case-ambiguous sentences. IWA showed deficits in using both types of morphological markers as a cue to non-canonical sentence interpretation and the results indicate that in aphasia, processing of case-marking cues is more vulnerable as compared to verb-agreement morphology. However, the online data revealed that IWA are in principle capable of successfully computing morphological cues, but the integration of morphological information is delayed as compared to age-matched controls. Furthermore, we found striking differences between controls and IWA regarding subject-before-object parsing predictions. While in case-unambiguous sentences IWA showed evidence for early subject-before-object parsing commitments, they exhibited no straightforward subject-first bias in case-ambiguous sentences, although controls did so for ambiguous structures. IWA delayed their parsing decisions in case-ambiguous sentences until unambiguous morphological information, such as a subject-verb-number-agreement cue, was available. We attribute the differential results for processing of case and agreement markers to differences in the degree of reliability of both morphological cues. We ascribe our findings for erroneous processing of case-unambiguous sentences in aphasia to late parsing commitments and failures in integrating case cues on time. For processing of case-ambiguous sentences in aphasia, we suggest that IWA adopt a wait-and-see strategy and make parsing commitments only when the agreement cue at the verb prompts a particular sentence structure. Our results for IWA further point to deficits in predictive processes during sentence comprehension.
Background: Individuals with aphasia (IWA) show deficits in comprehending object-extracted declaratives while comprehension of subject-extracted structures is relatively preserved. It is a matter of debate whether this subject–object asymmetry also arises for comprehension of wh-questions. Successful comprehension of wh-questions critically entails correct resolution of a filler–gap dependency. Most previous studies have used only offline accuracy measures to investigate wh-question comprehension in aphasia. Online studies exploring syntactic processing in real time are needed in order to draw inferences about gap-filling abilities in IWA and to identify the point of breakdown in sentence comprehension. Aims: This study aimed at investigating processing of subject and object who-questions in German-speaking IWA and in a group of controls by combining an offline and online method. We further aimed to explore the impact of case-marking cues on processing of wh-questions. Methods & Procedures: Applying a variant of the visual world eye-tracking paradigm, we measured participants’ eye movements while they performed the same offline task, which is frequently used to assess comprehension of declaratives (sentence–picture matching). Outcomes & Results: Concerning online processing of who-questions in controls, we found anticipation of the most likely post-verbal theta-role immediately after processing the case-marked wh-pronoun in both subject and object questions. In addition, we observed an unexpected advantage of object over subject questions in terms of processing time. The offline results for IWA revealed that there were three heterogeneous patterns: (a) symmetrical comprehension with equal impairments for both question types, (b) asymmetrical performance with better comprehension of subject than object who-questions, and (c) a reversed asymmetry with better comprehension of object as compared to subject questions. For online processing of both types of who-questions, IWA showed retained abilities in postulating the gap and in associating the filler with this gap, although they were slower as compared to controls. Moreover, similarly to controls, they anticipated the most likely post-verbal theta-role. Conclusions: For controls, the findings provide evidence for rapid resolution of the filler–gap dependency and incremental processing of case-marking cues, reflected in early prediction of upcoming syntactic structure. We attribute faster processing of object questions to faster alignment of the anticipated element with a semantically more salient character. For IWA, the online data provide evidence for retained predictive abilities in processing of filler–gap dependencies in wh-questions, but prediction was delayed. This is most likely attributed to delayed integration of case-marking cues.
This is the first attempt at characterizing reading difficulty in Hindi using naturally occurring sentences. We created the Potsdam-Allahabad Hindi Eyetracking Corpus by recording eye-movement data from 30 participants at the University of Allahabad, India. The target stimuli were 153 sentences selected from the beta version of the Hindi-Urdu treebank. We find that word- or low-level predictors (syllable length, unigram and bigram frequency) affect first-pass reading times, regression path duration, total reading time, and outgoing saccade length. An increase in syllable length results in longer fixations, and an increase in word unigram and bigram frequency leads to shorter fixations. Longer syllable length and higher frequency lead to longer outgoing saccades. We also find that two predictors of sentence comprehension diffi- culty, integration and storage cost, have an effect on reading difficulty. Integration cost (Gibson, 2000) was approximated by calculating the distance (in words) between a dependent and head; and storage cost (Gibson, 2000), which measures difficulty of maintaining predictions, was estimated by counting the number of predicted heads at each point in the sentence. We find that integration cost mainly affects outgoing saccade length, and storage cost affects total reading times and outgoing saccade length. Thus, word-level predictors have an effect in both early and late measures of reading time, while predictors of sentence comprehension difficulty tend to affect later measures. This is, to our knowledge, the first demonstration using eye-tracking that both integration and storage cost influence reading difficulty.
Chinese relative clauses are an important test case for pitting the predictions of expectation-based accounts against those of memory-based theories. The memory-based accounts predict that object relatives are easier to process than subject relatives because, in object relatives, the distance between the relative clause verb and the head noun is shorter. By contrast, expectation-based accounts such as surprisal predict that the less frequent object relative should be harder to process. In previous studies on Chinese relative clause comprehension, local ambiguities may have rendered a comparison between relative clause types uninterpretable. We designed experimental materials in which no local ambiguities confound the comparison. We ran two experiments (self-paced reading and eye-tracking) to compare reading difficulty in subject and object relatives which were placed either in subject or object modifying position. The evidence from our studies is consistent with the predictions of expectation-based accounts but not with those of memory-based theories.
Two classes of account have been proposed to explain the memory processes subserving the processing of reflexive-antecedent dependencies. Structure-based accounts assume that the retrieval of the antecedent is guided by syntactic tree-configurational information without considering other kinds of information such as gender marking in the case of English reflexives. By contrast, unconstrained cue-based retrieval assumes that all available information is used for retrieving the antecedent. Similarity-based interference effects from structurally illicit distractors which match a non-structural retrieval cue have been interpreted as evidence favoring the unconstrained cue-based retrieval account since cue-based retrieval interference from structurally illicit distractors is incompatible with the structure-based account. However, it has been argued that the observed effects do not necessarily reflect interference occurring at the moment of retrieval but might equally well be accounted for by interference occurring already at the stage of encoding or maintaining the antecedent in memory, in which case they cannot be taken as evidence against the structure-based account. We present three experiments (self-paced reading and eye-tracking) on German reflexives and Swedish reflexive and pronominal possessives in which we pit the predictions of encoding interference and cue-based retrieval interference against each other. We could not find any indication that encoding interference affects the processing ease of the reflexive-antecedent dependency formation. Thus, there is no evidence that encoding interference might be the explanation for the interference effects observed in previous work. We therefore conclude that invoking encoding interference may not be a plausible way to reconcile interference effects with a structure-based account of reflexive processing.
We conducted two eye-tracking experiments investigating the processing of the Mandarin reflexive ziji in order to tease apart structurally constrained accounts from standard cue-based accounts of memory retrieval. In both experiments, we tested whether structurally inaccessible distractors that fulfill the animacy requirement of ziji influence processing times at the reflexive. In Experiment 1, we manipulated animacy of the antecedent and a structurally inaccessible distractor intervening between the antecedent and the reflexive. In conditions where the accessible antecedent mismatched the animacy cue, we found inhibitory interference whereas in antecedent-match conditions, no effect of the distractor was observed. In Experiment 2, we tested only antecedent-match configurations and manipulated locality of the reflexive-antecedent binding (Mandarin allows non-local binding). Participants were asked to hold three distractors (animate vs. inanimate nouns) in memory while reading the target sentence. We found slower reading times when animate distractors were held in memory (inhibitory interference). Moreover, we replicated the locality effect reported in previous studies. These results are incompatible with structure-based accounts. However, the cue-based ACT-R model of Lewis and Vasishth (2005) cannot explain the observed pattern either. We therefore extend the original ACT-R model and show how this model not only explains the data presented in this article, but is also able to account for previously unexplained patterns in the literature on reflexive processing.
Traxler et al. (1998) found that ambiguous sentences are read faster than their unambiguous counterparts. This so-called ambiguity advantage has presented a major challenge to classical theories of human sentence comprehension (parsing) because its most prominent explanation, in the form of the unrestricted race model (URM), assumes that parsing is non-deterministic. Recently, Swets et al. (2008) have challenged the URM. They argue that readers strategically underspecify the representation of ambiguous sentences to save time, unless disambiguation is required by task demands. When disambiguation is required, however, readers assign sentences full structure — and Swets et al. provide experimental evidence to this end. On the basis of their findings they argue against the URM and in favor of a model of task-dependent sentence comprehension. We show through simulations that the Swets et al. data does not constitute evidence for task-dependent parsing because it can be explained by the URM. However, we provide decisive evidence from a German self-paced reading study consistent with Swets et al.’s general claim about task-dependent parsing. Specifically, we show that under certain conditions, ambiguous sentences can be read more slowly than their unambiguous counterparts, suggesting that the parser may create several parses, when required. Finally, we present the first quantitative model of task-driven disambiguation which subsumes the URM, and show that it can explain both Swets et al.’s results and our findings.
Swets et al. (2008) present evidence that the so-called ambiguity advantage (Traxler et al., 1998), which has been explained in terms of the Unrestricted Race Model, can equally well be explained by assuming underspecification in ambiguous conditions driven by task-demands. Specifically, if comprehension questions require that ambiguities be resolved, the parser tends to make an attachment: when questions are about superficial aspects of the target sentence, readers tend to pursue an underspecification strategy. It is reasonable to assume that individual differences in strategy will play a significant role in the application of such strategies, so that studying average behavior may not be informative. In order to study the predictions of the good-enough processing theory, we implemented two versions of underspecification: the partial specification model (PSM), which is an implementation of the Swets et al. proposal, and a more parsimonious version, the non-specification model (NSM). We evaluate the relative fit of these two kinds of underspecification to Swets et al.’s data; as a baseline, we also fit three models that assume no underspecification. We find that a model without unspecification pro- vides a somewhat better fit than both underspecification models, while the NSM model provides a better fit than the PSM. We interpret the results as lack of unambiguous evidence in favor of underspecification; however, given that there is considerable existing evidence for good-enough processing in the literature, it is reasonable to assume that some underspecification might occur. Under this assumption, the results can be interpreted as tentative ev- idence for NSM over PSM. More generally, our work provides a method for choosing between models of real-time processes in sentence comprehension that make qualitative predictions about the relationship between several de- pendent variables. We believe that sentence processing research will greatly benefit from a wider use of such methods.
There is a wealth of evidence showing that increasing the distance between an argument and its head leads to more processing effort, namely, locality effects; these are usually associated with constraints in working memory (DLT: Gibson, 2000; activation-based model: Lewis and Vasishth, 2005). In SOV languages, however, the opposite effect has been found: antilocality (see discussion in Levy et al., 2013). Antilocality effects can be explained by the expectation-based approach as proposed by Levy (2008) or by the activation-based model of sentence processing as proposed by Lewis and Vasishth (2005). We report an eye-tracking and a self-paced reading study with sentences in Spanish together with measures of individual differences to examine the distinction between expectation- and memory-based accounts, and within memory-based accounts the further distinction between DLT and the activation-based model. The experiments show that (i) antilocality effects as predicted by the expectation account appear only for high-capacity readers; (ii) increasing dependency length by interposing material that modifies the head of the dependency (the verb) produces stronger facilitation than increasing dependency length with material that does not modify the head; this is in agreement with the activation-based model but not with the expectation account; and (iii) a possible outcome of memory load on low-capacity readers is the increase in regressive saccades (locality effects as predicted by memory-based accounts) or, surprisingly, a speedup in the self-paced reading task; the latter consistent with good-enough parsing (Ferreira et al., 2002). In sum, the study suggests that individual differences in working memory capacity play a role in dependency resolution, and that some of the aspects of dependency resolution can be best explained with the activation-based model together with a prediction component.
SOPARSE (Tabor & Hutchins, 2004) predicts so-called local coherence effects: locally plausible but globally impossible parses of substrings can exert a distracting influence during sentence processing. Additionally, it predicts digging-in effects: the longer the parser stays committed to a particular analysis, the harder it becomes to inhibit that analysis. We investigated the interaction of these two predictions using German sentences. Results from a self-paced reading study show that the processing difficulty caused by a local coherence can be reduced by first allowing the globally correct parse to become entrenched, which supports SOPARSE’s assumptions.
Individuals with agrammatic Broca's aphasia experience difficulty when processing reversible non-canonical sentences. Different accounts have been proposed to explain this phenomenon. The Trace Deletion account attributes this deficit to an impairment in syntactic representations, whereas others propose that the underlying structural representations are unimpaired, but sentence comprehension is affected by processing deficits, such as slow lexical activation, reduction in memory resources, slowed processing and/or intermittent deficiency, among others. We test the claims of two processing accounts, slowed processing and intermittent deficiency, and two versions of the Trace Deletion Hypothesis, in a computational framework for sentence processing implemented in ACT-R. The assumption of slowed processing is operationalized as slow procedural memory, so that each processing action is performed slower than normal, and intermittent deficiency as extra noise in the procedural memory, so that the parsing steps are more noisy than normal. We operationalize the Trace Deletion Hypothesis as an absence of trace information in the parse tree. To test the predictions of the models implementing these theories, we use the data from a German sentence-picture matching study reported in Hanne-EtAl-2011. The data consists of offline (sentence-picture matching accuracies and response times) and online (eye fixation proportions) measures. From among the models considered, the model assuming that both slowed processing and intermittent deficiency are present emerges as the best model of sentence processing difficulty in aphasia. The modeling of individual differences suggests that, if we assume that patients have both slowed processing and intermittent deficiency, they have them in differing degrees.
Scanpaths have played an important role in classic research on reading behavior. Nevertheless, they have largely been neglected in later research—perhaps also due to a lack of suitable analytical tools. Recently, von der Malsburg and Vasishth (2011) proposed a new measure for quantifying differences between scanpaths and demonstrated that this measure can recover effects that were missed with the traditional eyetracking measures. However, the sentences used in that study were difficult to process and scanpath effects accordingly strong. The purpose of the present study was to test the validity, sensitivity, and scope of applicability of the scanpath measure using simple sentences that are typically read straight from left to right. We derived predictions for the regularity of scanpaths from the literature on oculomotor control, sentences processing, and cognitive aging and tested these predictions using the scanpath measure and a large database of eye movements (N=230). All predictions were confirmed: sentences with short words and syntactically more difficult sentences elicited more irregular scanpaths. Also, older readers produced more irregular scanpaths than younger readers. In addition, we found an effect that was not reported earlier: syntax had a smaller influence on the eye movements of older readers than on those of young readers. We discuss this interaction of syntactic parsing cost with age in terms of shifts in processing strategies and a decline of executive control as readers age. Overall, our results demonstrate the validity and sensitivity of the scanpath measure and thus establish it as a productive and versatile tool for reading research.
In explicit memory recall and recognition tasks, elaboration and contextual isolation both facilitate memory performance. Here, we investigate these effects in the context of sentence processing: targets for retrieval during online sentence processing of English object relative clause constructions differ in the amount of elaboration associated with the target noun phrase, or the homogeneity of superficial features (text color). Experiment 1 shows that greater elaboration for targets during the encoding phase reduces reading times at retrieval sites, but elaboration of non-targets has considerably weaker effects. Experiment 2 illustrates that processing isolated superficial features of target noun phrases—here, a green word in a sentence with words colored white—does not lead to enhanced memory performance, despite triggering longer encoding times. These results are interpreted in the light of the memory models of Nairne, 1990, 2001, 2006, which state that encoding remnants contribute to the set of retrieval cues that provide the basis for similarity-based interference effects.
Expectation-driven facilitation (Hale, 2001; Levy, 2008) and locality-driven retrieval difficulty (Gibson, 1998, 2000; Lewis & Vasishth, 2005) are widely recognized to be two critical factors in incremental sentence processing; there is accumulating evidence that both can influence processing difficulty. However, it is unclear whether and how expectations and memory interact. We first confirm a key prediction of the expectation account: a Hindi self-paced reading study shows that when an expectation for an upcoming part of speech is dashed, building a rarer structure consumes more processing time than building a less rare structure. This is a strong validation of the expectation-based account. In a second study, we show that when expectation is strong, i.e., when a particular verb is predicted, strong facilitation effects are seen when the appearance of the verb is delayed; however, when expectation is weak, i.e., when only the part of speech “verb” is predicted but a particular verb is not predicted, the facilitation disappears and a tendency towards a locality effect is seen. The interaction seen between expectation strength and distance shows that strong expectations cancel locality effects, and that weak expectations allow locality effects to emerge.
Recent research has shown that brain potentials time-locked to fixations in natural reading can be similar to brain potentials recorded during rapid serial visual presentation (RSVP). We attempted two replications of Hagoort, Hald, Bastiaansen, and Petersson [Hagoort, P., Hald, L., Bastiaansen, M., & Petersson, K. M. Integration of word meaning and world knowledge in language comprehension. Science, 304, 438–441, 2004] to determine whether this correspondence also holds for oscillatory brain responses. Hagoort et al. reported an N400 effect and synchronization in the theta and gamma range following world knowledge violations. Our first experiment (n = 32) used RSVP and replicated both the N400 effect in the ERPs and the power increase in the theta range in the time–frequency domain. In the second experiment (n = 49), participants read the same materials freely while their eye movements and their EEG were monitored. First fixation durations, gaze durations, and regression rates were increased, and the ERP showed an N400 effect. An analysis of time–frequency representations showed synchronization in the delta range (1–3 Hz) and desynchronization in the upper alpha range (11–13 Hz) but no theta or gamma effects. The results suggest that oscillatory EEG changes elicited by world knowledge violations are different in natural reading and RSVP. This may reflect differences in how representations are constructed and retrieved from memory in the two presentation modes.
A general fact about language is that subject relative clauses are easier to process than object relative clauses. Recently, several self-paced reading studies have presented surprising evidence that object relatives in Chinese are easier to process than subject relatives. We carried out three self-paced reading experiments that attempted to replicate these results. Two of our three studies found a subject-relative preference, and the third study found an object-relative advantage. Using a random effects Bayesian meta-analysis of fifteen studies (including our own), we show that the overall current evidence for the subject-relative advantage is quite strong (approximate posterior probability of a subject-relative advantage given the data: 78–80%). We argue that retrieval/integration based accounts would have difficulty explaining all three experimental results. These findings are important because they narrow the theoretical space by limiting the role of an important class of explanation—retrieval/integration cost—at least for relative clause processing in Chinese.
We explore the interaction between oculomotor control and language comprehension on the sentence level using two well-tested computational ac- counts of parsing difficulty. Previous work (Boston, Hale, Vasishth, & Kliegl, 2011) has shown that surprisal (Hale, 2001; Levy, 2008) and cue-based mem- ory retrieval (Lewis & Vasishth, 2005) are significant and complementary predictors of reading time in an eyetracking corpus. It remains an open question how the sentence processor interacts with oculomotor control. Using a simple linking hypothesis proposed in Reichle, Warren, and McConnell (2009), we integrated both measures with the eye movement model EMMA (Salvucci, 2001) inside the cognitive architecture ACT-R (Anderson et al., 2004). We built a reading model that could initiate short “Time Out regressions” (Mitchell, Shen, Green, & Hodgson, 2008) that compensate for slow postlexical processing. This simple interaction enabled the model to predict the re-reading of words based on parsing difficulty. The model was evaluated in different configurations on the prediction of frequency effects on the Potsdam Sentence Corpus. The extension of EMMA with postlexical processing improved its predictions and reproduced re-reading rates and durations with a reasonable fit to the data. This demonstration, based on simple and independently motivated assumptions, serves as a foundational step toward a precise investigation of the interaction between high-level language processing and eye movement control.
In Persian, a construction exists in which a gap can optionally be replaced by an overt pronoun. A self-paced reading study (110 participants) suggests that the overt pronoun results in deeper encoding (higher activation) of the antecedent noun, presumably because of richer retrieval cue specifications during antecedent retrieval at the pronoun; this higher activation has the consequence that the antecedent is easier to retrieve at a subsequent stage. This provides new evidence for reactivation effects of the type assumed in the cue-based retrieval model of parsing (Lewis and Vasishth, 2005), and shows that dependency resolution is not simply a matter of connecting two co-dependents; the retrieval cue specification has a differential impact on processing.
Eye-movement research on implicit prosody has found effects of lexical stress on syntactic ambiguity resolution, suggesting that metrical well-formedness constraints interact with syntactic category assignment. Building on these findings, the present eyetracking study investigates whether contextual bias can modulate the effects of metrical structure on syntactic ambiguity resolution in silent reading. Contextual bias and potential stress-clash in the ambiguous region were crossed in a 2 × 2 design. Participants read biased context sentences followed by temporarily ambiguous test sentences. In the three-word ambiguous region, main effects of lexical stress were dominant, while early effects of context were absent. Potential stress clash yielded a significant increase in first-pass regressions and re-reading probability across the three words. In the disambiguating region, the disambiguating word itself showed increased processing difficulty (lower skipping and increased re-reading probability) when the disambiguation engendered a stress clash configuration, while the word immediately following showed main effects of context in those same measures. Taken together, effects of lexical stress upon eye movements were swift and pervasive across first-pass and second-pass measures, while effects of context were relatively delayed. These results indicate a strong role for implicit meter in guiding parsing, one that appears insensitive to higher-level constraints. Our findings are problematic for two classes of models, the two-stage garden-path model and the constraint-based competition-integration model, but can be explained by a variation on the two-stage model, the unrestricted race model.
What theories best characterize the parsing processes triggered upon encountering ambiguity, and what effects do these processes have on eye movement patterns in reading? The present eye-tracking study, which investigated processing of attachment ambiguities of an adjunct in Spanish, suggests that readers sometimes underspecify attachment to save memory resources, consistent with the good-enough account of parsing. Our results confirm a surprising prediction of the good-enough account: high-capacity readers commit to an attachment decision more often than low-capacity participants, leading to more errors and a greater need to reanalyze in garden-path sentences. These results emerged only when we separated functionally different types of regressive eye movements using a scanpath analysis; conventional eye-tracking measures alone would have led to different conclusions. The scanpath analysis also showed that rereading was the dominant strategy for recovering from garden-pathing. Our results may also have broader implications for models of reading processes: reanalysis effects in eye movements occurred late, which suggests that the coupling of oculo-motor control and the parser may not always be as tight as assumed in current computational models of eye movements control in reading.
Background: In behavioural tests of sentence comprehension in aphasia, correct and incorrect responses are often randomly distributed. Such a pattern of chance performance is a typical trait of Broca’s aphasia, but can be found in other aphasic syndromes as well. Many researchers have argued that chance behaviour is the result of a guessing strategy, which is adopted in the face of a syntactic breakdown in sentence processing. Aims: Capitalising on new evidence from recent studies investigating online sentence comprehension in aphasia using the visual world paradigm, the aim of this paper is to review the concept of chance performance as a reflection of a syntactic impairment in sentence processing and to re-examine the conventional interpretation of chance performance as a guessing behaviour. Main Contribution: Based on a review of recent evidence from visual world paradigm studies, we argue that the assumption of chance performance equalling guessing is not necessarily compatible with actual real-time parsing procedures in people with aphasia. We propose a reinterpretation of the concept of chance performance by assuming that there are two distinct processing mechanisms underlying sentence comprehension in aphasia. Correct responses are always the result of normal-like parsing mechanisms, even in those cases where the overall performance pattern is at chance. Incorrect responses, on the other hand, are the result of intermittent deficiencies of the parser. Hence the random guessing behaviour that persons with aphasia often display does not necessarily reflect a syntactic breakdown in sentence comprehension and a random selection between alternatives. Instead it should be regarded as a result of temporal deficient parsing procedures in otherwise normal-like comprehension routines. Conclusion: Our conclusion is that the consideration of behavioural offline data alone may not be sufficient to interpret a performance in language tests and subsequently draw theoretical conclusions about language impairments. Rather it is important to call on additional data from online studies that look at language processing in real time in order to gain a comprehensive picture about syntactic comprehension abilities of people with aphasia and possible underlying deficits.
Two Hindi eyetracking studies show that clefting a noun results in greater processing difficulty initially, due to the extra processing steps involved in encoding a clefted noun (e.g., for computing the exhaustiveness interpretation). However, this extra difficulty in encoding a clefted noun results in a processing advantage when the clefted noun needs to be retrieved later on in the sentence - the clefted noun is retrieved faster in subsequent processing compared to its non-clefted counterpart. This effect is short-lived, however; it does not last beyond the current sentence. We also show that given-new ordering yields a processing advantage over new-given order, but this is only seen after the whole sentence is processed, i.e., it is a late effect that occurs after syntactic processing is completed. Finally, following up on work on German by Hoernig et al. (2005), we present evidence that non-canonical order can be processed more easily than canonical order given appropriate context.
Eye movement data have proven to be very useful for investigating human sentence processing.Eyetracking research has addressed a wide range of questions, such as recovery mechanisms following garden-pathing, the timing of processes driving comprehension, the role of anticipation and expectation in parsing, the role of semantic, pragmatic, and prosodic information, and so on. However, there are some limitations regarding the inferences that can be made on the basis of eye movements. One relates to the nontrivial interaction between parsing and the eye movement control system which complicates the interpretation of eye movement data. Detailed computational models that integrate parsing with eye movement control theories have the potential to unpack the complexity of eyemovement data and can therefore aid in the interpretation of eye movements. Another limitation is the difficulty of capturing spatiotemporal patterns in eye movements using the traditional word-based eyetracking measures. Recent research has demonstrated the relevance of these patterns and has shown how they can be analyzed. In this review, we focus on reading, and present examples demonstrating how eye movement data reveal what events unfold when the parser runs into difficulty, and how the parsing system interacts with eye movement control
Many comprehension theories assert that increasing the distance between elements participating in a linguistic relation (e.g., a verb and a noun phrase argument) increases the difficulty of establishing that relation during on-line comprehension. Such locality effects are expected to increase reading times and are thought to reveal properties and limitations of the short-term memory system that supports comprehension. Despite their theoretical importance and putative ubiquity, however, evidence for on-line locality effects is quite narrow linguistically and methodologically: It is restricted almost exclusively to self-paced reading of complex structures involving a particular class of syntactic relation. We present 4 experiments (2 self-paced reading and 2 eyetracking experiments) that demonstrate locality effects in the course of establishing subject-verb dependencies; locality effects are seen even in materials that can be read quickly and easily. These locality effects are observable in the earliest possible eye-movement measures and are of much shorter duration than previously reported effects. To account for the observed empirical patterns, we outline a processing model of the adaptive control of button pressing and eye movements. This model makes progress toward the goal of eliminating linking assumptions between memory constructs and empirical measures in favor of explicit theories of the coordinated control of motor responses and parsing.
Eye fixation durations during normal reading correlate with processing difficulty, but the specific cognitive mechanisms reflected in these measures are not well understood. This study finds support in German readers‚Äô eye fixations for two distinct difficulty metrics: surprisal, which reflects the change in probabilities across syntactic analyses as new words are integrated; and retrieval, which quantifies comprehension difficulty in terms of working memory constraints.We examine the predictions of both metrics using a family of dependency parsers indexed by an upper limit on the number of candidate syntactic analyses they retain at successive words. Surprisal models all fixation measures and regression probability. By contrast, retrieval does not model any measure in serial processing. As more candidate analyses are considered in parallel at each word, retrieval can account for the same measures as surprisal. This pattern suggests an important role for ranked parallelism in theories of sentence comprehension.
While it is widely acknowledged in the formal semantic literature that both the truth-functional focus particle only and it-clefts convey exhaustiveness, the nature and source of exhaustiveness effects with it-clefts remain contested. Based on an event-related brain potentials (ERPs) study on only-foci and it-clefts, we provide experimental evidence that the violation or cancelation of exhaustive readings involve different underlying processes in the two structural environments.
Background: In addition to the canonical subject-verb-object (SVO) word order, German also allows for non-canonical order (OVS), and the case-marking system supports thematic role interpretation. Previous eye-tracking studies (Kamide et al., 2003; Knoeferle, 2007) have shown that unambiguous case information in non-canonical sentences is processed incrementally. For individuals with agrammatic aphasia, comprehension of non-canonical sentences is at chance level (Burchert et al., 2003). The trace deletion hypothesis (Grodzinsky 1995, 2000) claims that this is due to structural impairments in syntactic representations, which force the individual with aphasia (IWA) to apply a guessing strategy. However, recent studies investigating online sentence processing in aphasia (Caplan et al., 2007; Dickey et al., 2007) found that divergences exist in IWAs' sentence-processing routines depending on whether they comprehended non-canonical sentences correctly or not, pointing rather to a processing deficit explanation. Aims: The aim of the current study was to investigate agrammatic IWAs' online and offline sentence comprehension simultaneously in order to reveal what online sentence-processing strategies they rely on and how these differ from controls' processing routines. We further asked whether IWAs' offline chance performance for non-canonical sentences does indeed result from guessing. Methods & Procedures: We used the visual-world paradigm and measured eye movements (as an index of online sentence processing) of controls (N = 8) and individuals with aphasia (N = 7) during a sentence-picture matching task. Additional offline measures were accuracy and reaction times. Outcomes & Results: While the offline accuracy results corresponded to the pattern predicted by the TDH, IWAs' eye movements revealed systematic differences depending on the response accuracy. Conclusions: These findings constitute evidence against attributing IWAs' chance performance for non-canonical structures to mere guessing. Instead, our results support processing deficit explanations and characterise the agrammatic parser as deterministic and inefficient: it is slowed down, affected by intermittent deficiencies in performing syntactic operations, and fails to compute reanalysis even when one is detected.
Three experiments (self-paced reading, eyetracking and an ERP study) show that in relative clauses, increasing the distance be tween the relativized noun and the relative-clause verb makes it more difficult to process the relative-clause verb (the so-called locality effect). This result is consistent with the predictions of several theories (Gibson 2000, Lewis and Vasishth 2005), and contradicts the recent claim (Levy 2008) that in relative-clause structures increasing argument-verb distance makes processing easier at the verb. Levy's expectation-based account predicts that the expectation for a verb becomes sharper as dis- tance is increased and therefore processing becomes easier at the verb. We argue that, in addition to expectation effects (which are seen in the eyetracking study in first-pass regression probability), processing load also increases with increasing distance. This contradicts Levy's claim that heightened expectation leads to lower processing cost. Dependency- resolution cost and expectation-based facilitation are jointly responsible for determining processing cost.
Seven experiments using self-paced reading and eyetracking suggest that omitting the middle verb in a double centre embedding leads to easier processing in English but leads to greater difficulty in German. One commonly accepted explanation for the English pattern‚Äîbased on data from offline acceptability ratings and due to Gibson and Thomas (1999)‚Äîis that working-memory overload leads the comprehender to forget the prediction of the upcoming verb phrase (VP), which reduces working-memory load. We show that this VP-forgetting hypothesis does an excellent job of explaining the English data, but cannot account for the German results. We argue that the English and German results can be explained by the parser's adaptation to the grammatical properties of the languages; in contrast to English, German subordinate clauses always have the verb in clause-final position, and this property of German may lead the German parser to maintain predictions of upcoming VPs more robustly compared to English. The evidence thus argues against language-independent forgetting effects in online sentence processing; working-memory constraints can be conditioned by countervailing influences deriving from grammatical properties of the language under study.
Which repair strategy does the language system deploy when it gets garden-pathed and what can regressive eye movements in reading tell us about reanalysis strategies? Several influential eye-tracking studies on syntactic reanalysis (Frazier & Rayner 1982, Meseguer et al 2002, Mitchell et al 2008) have examined scanpaths-sequences of eye fixations-to answer this question. However, in the absence of a suitable method for analyzing scanpaths, these studies relied on simplified dependent measures that are arguably ambiguous and hard to interpret. We address the theoretical question of repair strategy by developing a new method that quantifies scanpath similarity. Our method reveals several distinct fixation strategies associated with reanalysis that went undetected in a previously published data set (Meseguer et al 2002). One prevalent pattern suggests re-parsing of the sentence, a strategy that has been proposed in the literature (Frazier & Rayner 1982); however, readers differed tremendously in how they orchestrated the various fixation strategies. Our results raise the possibility that the human parsing system non-deterministically adopts different strategies when confronted with the need to reanalyze.
This paper presents the results of an experimental study on multiple focus configurations, that is, structures containing two nested focus-sensitive operators plus two foci supposed to associate with those operators. There has been controversial discussion in the semantic literature regarding whether or not an interpretation is acceptable that corresponds to this association. While the data are unclear, the issue is of considerable theoretical significance, as it distinguishes between the available theories of focus interpretation. Some theories (e.g. Rooth, 1992) predict such a pattern of association with focus to be impossible, while others (such as Wold, 1996) predict it to be acceptable. The results of our study show the data to be unacceptable rather than acceptable, favouring important aspects of the theory of focus interpretation developed by Rooth.
A production study is presented that investigates the effects of word order and information structural context on the prosodic realization of declarative sentences in Hindi. Previous work on Hindi intonation has shown that: (i) non-final content words bear rising pitch accents (Moore 1965, Dyrud 2001, Nair 1999); (ii) focused constituents show greater pitch excursion and longer duration and that post-focal material undergoes pitch range reduction (Moore 1965, Harnsberger 1994, Harnsberger and Judge 1996); and (iii) focused constituents may be followed by a phrase break (Moore 1965). By means of a controlled experiment, we investigated the effect of focus in relation to word order variation using 1200 utterances produced by 20 speakers. Fundamental frequency (F0) and duration of constituents were measured in Subject-Object-Verb (SOV) and Object-Subject-Verb (OSV) sentences in different information structural conditions (wide focus, subject focus and object focus). The analyses indicate that (i) regardless of word order and focus, the constituents are in a strict downstep relationship; (ii) focus is mainly characterized by post-focal pitch range reduction rather than pitch raising of the element in focus; (iii) given expressions that occur pre-focally appear to undergo no reduction; (iv) pitch excursion and duration of the constituents is higher in OSV compared to SOV sentences. A phonological analysis suggests that focus affects pitch scaling and that word order influences prosodic phrasing of the constituents.
The surprisal of a word on a probabilistic grammar constitutes a promising complexity metric for human sentence comprehension difficulty. Using two different grammar types, surprisal is shown to have an effect on fixation durations and regression probabilities in a sample of German readers' eye movements, the Potsdam Sentence Corpus. A linear mixed-effects model was used to quantify the effect of surprisal while taking into account unigram frequency and bigram frequency (transitional probability), word length, and empirically-derived word predictability; the so-called ‚“early” and “late” measures of processing difficulty both showed an effect of surprisal. Surprisal is also shown to have a small but statistically non-significant effect on empirically-derived predictability itself. This work thus demonstrates the importance of including parsing costs as a predictor of comprehension difficulty in models of reading, and suggests that a simple identification of syntactic parsing costs with early measures and late measures with durations of post-syntactic events may be difficult to uphold.
A central question in online human sentence comprehension is: how are linguistic relations established between different parts of a sentence? Previous work has shown that this dependency resolution process can be computationally expensive, but the underlying reasons for this are still unclear. We argue that dependency resolution is mediated by cue-based retrieval, constrained by independently motivated working memory principles defined in a cognitive architecture (ACT-R). To demonstrate this, we investigate an unusual instance of dependency resolution, the processing of negative and positive polarity items, and confirm a surprising prediction of the cue-based retrieval model: partial cue-matches, which constitute a kind of similarity-based interference, can give rise to the intrusion of ungrammatical retrieval candidates, leading to both processing slow-downs and even errors of judgment that take the form of illusions of grammaticality in patently ungrammatical structures. A notable achievement is that good quantitative fits are achieved without adjusting the key model parameters.
Understanding a sentence requires a working memory of the partial products of comprehension, so that linguistic relations between temporally distal parts of the sentence can be rapidly computed. We describe an emerging theoretical framework for this working memory system that incorporates several independently motivated principles of memory: a sharply limited attentional focus, rapid retrieval of item (but not order) information subject to interference from similar items, and activation decay (forgetting over time). A computational model embodying these principles provides an explanation of the functional capacities and severe limitations of human processing, as well as accounts of reading times. The broad implication is that the detailed nature of crosslinguistic sentence processing emerges from the interaction of general principles of human memory with the specialized task of language comprehension.
Although proximity between arguments and verbs (locality) is a relatively robust determinant of sentence-processing difficulty (Hawkins 1998, 2001, Gibson 2000), increasing argument-verb distance can also facilitate processing (Konieczny 2000). We present two self-paced reading (SPR) experiments involving Hindi that provide further evidence of antilocality, and a third SPR experiment which suggests that similarity-based interference can attenuate this distance-based facilitation. A unified explanation of interference, locality, and antilocality effects is proposed via an independently motivated theory of activation decay and retrieval interference (Anderson et al. 2004).
We present a detailed process theory of the moment-by-moment working-memory retrievals and associated control structure that subserve sentence comprehension. The theory is derived from the application of independently motivated principles of memory and cognitive skill to the specialized task of sentence parsing. The resulting theory construes sentence processing as a series of skilled associative memory retrievals modulated by similarity-based interference and fluctuating activation. The cognitive principles are formalized in computational form in the Adaptive Control of Thought-Rational (ACT-R) architecture, and our process model is realized in ACT-R.We present the results of 6 sets of simulations: 5 simulation sets provide quantitative accounts of the effects of length and structural interference on both unambiguous and garden-path structures. A final simulation set provides a graded taxonomy of double center embeddings ranging from relatively easy to extremely difficult. The explanation of center- embedding difficulty is a novel one that derives from the model‚Äôs complete reliance on discriminating retrieval cues in the absence of an explicit representation of serial order information. All fits were obtained with only 1 free scaling parameter fixed across the simulations; all other parameters were ACT-R defaults. The modeling results support the hypothesis that fluctuating activation and similarity-based interference are the key factors shaping working memory in sentence processing. We contrast the theory and empirical predictions with several related accounts of sentence-processing complexity.
Discourse context has been argued to be the main factor responsible for increased processing difficulty in non-canonical order sentences: if appropriate discourse context is provided (the argument goes) both canonical and non-canonical order sentences are equally easy to process. This research suggests that this generalization may not be true across languages: the distance between arguments and verbs could affect the ease with which the former can be integrated with the latter, and sufficiently increasing this distance makes processing difficult, regardless of discourse context.
In Hindi certain word order possibilities that are grammatical in non-negative sentences become ungrammatical in the presence of sentential negation. In movement-based accounts of such negation-induced word order constraints, the restricted word order has been argued to provide evidence that negative polarity items (NPIs) in Hindi are licensed at LF and S-structure while in English NPI licensing occurs at S-structure. I argue for a non-movement-based, uniformly monostratal (S-structure) account for the word order facts in Hindi, cast in the multimodal categorial grammar framework. The NPI licensing issue is dealt with independently following Dowty's monotonicity marking analysis.
Unlike molecules or plots of barley, subjects in psycholinguistic experiments are intelligent beings that depend for their survival on constant adaptation to their environment. This study presents three data sets documenting the presence of adaptive processes in psychological experiments. These adaptive processes leave a statistical footprint in the form of autocorrelations in the residual error associated with by-subject time series of trial-to-trial responses. Generalized additive mixed models (GAMMs) provide a unified framework within which both factorial predictors and covariates given with the experimental design, as well as non-linear random effects and interactions with experimental time can be uncovered and evaluated. GAMMs not only provide substantially improved fits to experimental data with time series structure, but also provide improved insight into predictors of theoretical interest, as well as a more refined window on the random effects structure. Our results challenge the standard advocated by Barr et al. (2013). The analytical cage of the maximal linear mixed model to which this standard confines the analyst is motivated by simulation studies which presuppose experimental data to be sterile, and free of any adaptive processes. However, when adaptive processes are present in real data, the simulation results of Barr et al. are no longer informative. For such data, the method of analysis cannot be purely design-driven, but must be in part driven by the data.
The analysis of experimental data with mixed-effects models requires decisions about the specification of the appropriate random-effects structure. Recently, Barr, et al 2013, recommended fitting `maximal' models with all possible random effect components included. Estimation of maximal models, however, may not converge. We show that failure to converge typically is not due to a suboptimal estimation algorithm, but is a consequence of attempting to fit a model that is too complex to be properly supported by the data, irrespective of whether estimation is based on maximum likelihood or on Bayesian hierarchical modeling with uninformative or weakly informative priors. Importantly, even under convergence, overparameterization may lead to uninterpretable models. We provide diagnostic tools for detecting overparameterization and guiding model simplification. Finally, we clarify that the simulations on which Barr et al. base their recommendations are atypical for real data. A detailed example is provided of how subject-related attentional fluctuation across trials may further qualify statistical inferences about fixed effects, and of how such nonlinear effects can be accommodated within the mixed-effects modeling framework.
We report a comprehensive literature review of retrieval interference in reflexive-antecedent dependencies, number agreement, and non-agreement subject-verb dependencies, and computationally evaluate the predictions of cue-based retrieval theory with reference to published results. A novel finding from the review and modeling is that, contrary to claims in the literature, results on number agreement are not entirely compatible with cue-based retrieval theory. We also show that the cue-based retrieval account in its current form cannot explain several reported interference effects, such as (i) speed-ups observed in presence of a syntactically unlicensed distractor when the correct dependent is a full match to the retrieval cues and (ii) slow-downs when the correct dependent only partially matches the retrieval cues. We demonstrate that these effects can be explained by two theoretical and independently motivated constructs: distractor prominence and cue confusion. The cue-based retrieval model is therefore extended to incorporate distractor prominence and cue confusion, and quantitative predictions are derived from this extended model. We show that the extended cue-based retrieval model provides a better explanation of published results than the classical retrieval account.
Understanding a sentence and integrating it into the discourse depends upon the identification of its focus, which, in spoken German, is marked by accentuation. In the case of written language, which lacks explicit cues to accent, readers have to draw on other kinds of information to determine the focus. We study the joint or interactive effects of two kinds of information that have no direct representation in print but have each been shown to be influential in the reader's text comprehension: i. the (low-level) rhythmic-prosodic structure that is based on the distribution of lexically stressed syllables, and ii. the (high-level) discourse context that is grounded in the memory of previous linguistic content. Systematically manipulating these factors, we examine the way readers resolve a syntactic ambiguity involving the scopally ambiguous focus operator auch (engl. `too') in both oral (Experiment 1) and silent reading (Experiment 2). The results of both experiments attest that discourse context and local linguistic rhythm conspire to guide the syntactic and, concomitantly, the focus-structural analysis of ambiguous sentences. We argue that reading comprehension requires the (implicit) assignment of accents according to the focus structure and that, by establishing a prominence profile, the implicit prosodic rhythm directly affects accent assignment.
Linear mixed-effects models have increasingly replaced mixed-model analyses of variance for statistical inference in factorial psycholinguistic experiments. The advantages of LMMs over ANOVAs, however, come at a cost: Setting up an LMM is not as straightforward as running an ANOVA. One simple option, when numerically possible, is to fit the full variance-covariance structure of random effects (the maximal model; Barr et al., 2013), presumably to keep Type I error down to the nominal α in the presence of random effects. Although it is true that fitting a model with only random intercepts may lead to higher Type I error, fitting a maximal model also has a cost: it can lead to a significant loss of power. We demonstrate this with simulations and suggest that for typical psychological and psycholinguistic data, models with a random effect structure that is supported by the data have optimal Type I error and power properties.
In sentence comprehension, it is well-known that processing cost increases with dependency distance (Gibson 2000, Lewis and Vasishth 2005); this often referred to as the locality effect. However, the expectation-based account (Hale 2001, Levy 2008) predicts that delaying the appearance of a verb renders it more predictable and therefore easier to process. Following up on previous work (Husain et al 2014), we investigated whether strengthening the expectation can increase facilitation at the verb even further. We operationalize strong expectation as prediction of the lexical entry for the verb; by contrast, weak expectation refers to the prediction of some upcoming verb phrase (these are the cases discussed by Levy 2008). We used Persian for this investigation. This language has a special construction called complex predicates, which are separable Noun-Verb configurations in which the verb (the precise lexical item) is highly predictable given the noun. In two self-paced reading experiments, we delayed the appearance of the verb by interposing a relative clause (Expt 1, 42 subjects) or a long PP (Expt 2, 40 subjects). As a control, we included a simple predicate (Noun-Verb) configuration; the same distance manipulation was applied here as for complex predicates, but here, the exact lexical entry for the verb is not predicted but rather a verb phrase is predicted. Thus, we had a 2x 2 design, with Expectation Strength (Strong/Weak) and Distance (Short/Long). Based on the Husain et al study, which had a similar design using Hindi complex predicates, we expected a slowdown in the weak expectation condition (i.e., locality effects), but a facilitation in the strong expectation conditions (i.e., expectation effects). Surprisingly, both experiments showed clear effects of locality in both the strong and weak expectation conditions. We also find evidence that could be consistent with expectation effects: the high-predictable verbs are read faster than the low-predictable verbs. However, this result is difficult to interpret because the verbs used in the strong and weak expectation conditions are different. In sum, these studies show strong and unequivocal evidence in favor of argument-verb dependency distance influencing integration processes at the verb, falsifying a key prediction of the expectation based account of Levy 2008.
With the arrival of the R packages nlme and lme4, linear mixed models (LMMs) have come to be widely used in experimentally-driven areas like psychology, linguistics, and cognitive science. This tutorial provides a practical introduction to fitting LMMs in a Bayesian framework using the probabilistic programming language Stan. We choose Stan (rather than WinBUGS or JAGS) because it provides an elegant and scalable framework for fitting models in most of the standard applications of LMMs. We ease the reader into fitting increasingly complex LMMs, first using a two-condition repeated measures self-paced reading study, followed by a more complex 2×2 repeated measures factorial design that can be generalized to much more complex designs.
How important is the ability to freely control eye movements for reading comprehension? We investigated this question using event-related brain potentials recorded while participants read either word-by-word (also known as RSVP) or naturally. Additionally, eye movements were recorded concurrently with brain potentials during natural reading. Word-by-word presentation and natural reading both elicited similar N400 and P600 effects in response to syntactic and semantic violations. However, comprehension accuracy was higher in natural reading than in word-by-word presentation and particularly high when participants regressed to earlier portions of the sentence after encountering the violation. A more fine-grained ERP analysis showed that P600 effects, which are believed to reflect recovery processes, only occurred in trials with regressive eye movements. In trials without regressions, we instead found either a sustained, centro-parietal negativity starting at around 320 ms post-onset or no effect depending on the position of the violation within the sentence. Thus, the combined analysis of eye movements and ERPs reveals that the sentence comprehension system engages in strategic choices when confronted with difficult material and that the ability to reread earlier parts of the sentence is the key to thorough comprehension.