The RNN explains variance that the PSG does not account for, but the reverse is not the case. Taking only content words, results are similar except that the RNN now outperforms the n-gram model. Effects on function words are very weak in general and, consequently, no one model type accounts for variance over and above any other. If a word (or its part-of-speech) conveys more information, it takes longer

to read the word. The first objective of the current study was to investigate whether ERP amplitude, too, depends on word and PoS information. Our expectation that the N400 would be related to word surprisal was indeed borne out. Other components and information measures, however, did not show any find more reliable correlation. Our second objective was to identify the model type whose information measures best predict the ERP data. Generally speaking, the Y-27632 n-gram and RNN models outperformed the PSG in this respect. Reading a word with higher surprisal value, under any of the three language model types, results in increased N400 amplitude. This finding confirms that the ERP component is sensitive to word predictability. Whereas previous studies (e.g., Dambacher et al., 2006, Kutas and Hillyard, 1984, Moreno et al.,

2002 and Wlotko and Federmeier, 2013) used subjective human ratings to quantify predictability, we operationalized (un)predictability as the information-theoretic concept of surprisal, as estimated by probabilistic language models that were trained on a large text corpus. Although word surprisal

can be viewed as a more formal variant of cloze probability, it was not obvious in advance that the known effect of cloze probability on N400 size could be replicated by surprisal. As Smith and Levy (2011) demonstrated, systematic differences exist between cloze and corpus-based word probabilities, and cloze probabilities appear to predict word reading-times more accurately. Across the full range of surprisal values, average N400 amplitudes differed by about 1 μV. Dambacher et al. (2006), too, found a difference of approximately Urease 1 μV between content words with lowest and highest cloze probability. Experiments in which only sentence-final words are varied typically result in much larger effect sizes, with N400 amplitude varying by about 4 μV between high- and low-cloze (but not semantically anomalous) words (Kutas and Hillyard, 1984 and Wlotko and Federmeier, 2013). Most likely, this is because effects are more pronounced on sentence-final words, or because cloze differences tend to be larger in hand-crafted experimental sentences than in our (and Dambacher et al.’s) naturalistic materials. All model types could account for the N400 effect as long as their linguistic accuracy was sufficient.