Lexical versus structural accounts of the Ding et al. results

The model’s only linguistic representations are formed by the distributional semantics vectors. Although these are learned from word strings (see Methods), they do not explicitly encode information about word sequences (such as transitional probabilities) and can therefore not be used to predict, for example, that a noun is likely to be followed by a verb [16]. More importantly, a word’s vector does not depend on its position or neighbors in the stimulus sentence, and the model lacks higher-level representations of phrases and sentences. This, of course, raises a crucial question: What is the origin of the predicted power peaks at the presentation rates of phrases and sentences?

The word vectors represent lexical properties by virtue of the fact that similarities between vectors mirror paradigmatic relations between words: Words that share more syntactic/semantic properties are encoded by more similar vectors. Consequently, if certain lexical properties occur at a fixed rate in the stimulus sequence, this will be reflected as a recurring approximate numerical pattern in the model’s time-series of vectors. For example, in Ding et al.’s English four-word sentences condition, every other word is a noun, most often referring to some entity, and every forth word is a transitive verb, usually referring to an action. Because semantically and syntactically similar words are represented by similar vectors, the vector sequence corresponding to this experimental condition shows spectral power peaks at exactly the occurrence rates of two-word phrases and four-word sentences. Crucially, this does not rely on any hierarchical structure or process: Vectors represent only lexical information and the spectral power analysis is applied over a sequence of vectors that is not processed, integrated, or interpreted. Note, however, that whether a word is a noun or verb (or, in terms of semantics: refers to an entity or action) depends on its role within the sentence. In the stimulus sentence “fat rat sensed fear”, for example, the individual word “fat” could be a noun instead of an adjective and “fear”, on its own, could be a verb instead of a noun. The vectors do not distinguish between the different senses of these ambiguous words: There is only one representation of “fat”. Nevertheless, there is apparently enough repetition in the stimulus word’s properties to account for the MEG results. In fact, we obtained qualitatively similar results when each word was represented by a vector that merely identifies its most frequent syntactic category, that is, independently of the word’s role in the sentence (see S1 Appendix).

The model shows how Ding et al.’s cortical entrainment results can be explained without recourse to hierarchical structures or integrative processes. However, this does not rule out that cortical entrainment to hierarchical structure exists or even that the Ding et al. results in fact do reflect processing of hierarchical syntax. Indeed, it has recently been demonstrated that a hierarchical sentence processing model predicts similar power spectra, at least on English materials [19]. Whether lexical or phrasal/sentential properties of the speech signal are in fact responsible for cortical entrainment can possibly be established by testing on a ‘word salad’ version of the English stimuli sequence, created by keeping all verbs and adjectives in place but replacing all nouns by random words from different syntactic categories. This breaks any (consistent) syntactic structure so no 1 Hz peak should be visible if its occurrence depends on the repeated presence of 4-word sentences. In contrast, our lexical model does predict 1 Hz power in this word-salad condition because both verbs and adjectives still occur at a 1 Hz rate (see S1 Fig).

Returning to the stimuli sequences tested by Ding et al., our model demonstrates that hierarchical syntax is not necessary to explain these entrainment results: Representing only lexical properties of the stimuli suffices. Compared to structural accounts (such as [19]), our lexical explanation has the advantage of parsimony. This is because constructing a sentence’s hierarchical structure requires information about the words’ possible syntactic categories (i.e., whether a word can be a noun, verb, or adjective); or in semantic terms: Understanding a sentence requires at least some knowledge of word meaning. Thus, lexical properties are necessary in a structural explanation of the entrainment results. Conversely, however, the vector representations in our model do not depend on the sentence in which the words happen to appear. Hence, the lexical model forms a more parsimonious account.

Word vectors as cortical representations

The vector representations are not intended to be neurally realistic, that is, we do not claim they are isomorphic to cortical representations. Rather, the relation between model and brain is a higher-order one: The rhythmic recurrence of patterns in a vector sequence corresponds to the patterns in the MEG signal caused by perceiving the stimuli represented by the vectors. Ding et al. report two additional experiments that probe the cortical representations more directly. First, they show that overall MEG activity strongly decreases around phrase and sentence boundaries. Second, they find a spatial dissociation between the cortical areas that show a phrasal- or sentential-rate response. Although there was no reason to expect our vectors to account for these findings too, we did find that vector lengths show a sharp drop right after phrase and sentence boundaries (S2 Appendix) and that (apparent) effects of phrases and sentences can be localized in different vector dimensions (S3 Appendix). Hence, the correspondence between the distributional semantic vectors and cortical representations may be stronger than anticipated.

Materials

All materials were taken directly from the Ding et al. experiments. For English, these were 60 four-word sentences with structure as in (1) except that in seven sentences, the first word was not an adjective but a numeral or possessive pronoun. All English words were monosyllabic. For Chinese, the written items provided by Ding et al. were converted into pinyin (a phonological representation of Mandarin Chinese) by Pypinyin (version 0.12.1; we replaced the package’s word-pinyin dictionary with a larger one from ZDIC) after which the result was manually checked and corrected when needed. Converting the written characters to pinyin is essential because the original experiments used auditory stimuli presentation. The same pinyin syllable can correspond to many different written characters, making word co-occurrence patterns differ strongly between spoken and written Chinese. Consequently, a distributional semantics model can only adequately capture the lexical information in spoken stimuli if it is applied to the pinyin form.

There were two sets of 50 four-syllable Chinese sentences in the Ding et al. experiments. Sentences in the first set consisted of a two-syllable noun and a two-syllable verb or verb phrase, for example “lǎoniú gēngdì”(老牛耕地; “Old cattle ploughs the field”). The second set of Chinese sentences consisted of a monosyllablic verb followed by a three-syllable noun or noun phrase, for instance, “zhēng guàntāngbāo”(蒸灌汤包; “Braising the soup dumplings”).

Following Ding et al., syllable strings containing only two-syllable verbs or verb phrases were constructed by taking the verb (phrase) parts from the [N V(P)] sentences. Furthermore, four-syllable sequences without any consistent word, phrase, or sentence structure were obtained by randomly reassigning syllables to the [N V(P)] sentences while retaining each syllable’s position in the sentence.

To summarize, there are five experimental conditions using the same stimuli as Ding et al.: one in English ([NP VP] sentences) and four in Chinese ([N V(P)] sentences, [V N(P)] sentences, verb (phrases) only, and shuffled syllable sequences). Depending on condition, each stimuli sequence is composed of either four-syllable sentences, two-syllable verbs or verb phrases, or individual syllables without further linguistic structure.

Representing lexical knowledge

Lexical information over time

In Ding et al.’s Chinese experiments, syllables come in at a fixed rate of 4 Hz, or every 250 ms. The English experiments used a slightly slower presentation rate, but for simplicity we model English and Chinese experiments using the same 4 Hz rate.

Let be the N-dimensional column vector that represents the English word or Chinese syllable sequence currently being presented. We assume that the lexical information does not immediately appear at word onset (t = 0 ms) but some time later, at τ ≥ 0. The value of τ is randomly sampled at each word/syllable presentation, from a uniform distribution with mean μ = 40 and width β = 50 (how this choice of parameter values came about is discussed below).

The available lexical information at t milliseconds after word onset (for 0 ≤ t ≤ 250) is represented by a column vector with where ε_i(t) denotes Gaussian noise with mean 0 and standard deviation σ = 0.5. Hence, the lexical information vector w(t) starts with representing only noise but at t = τ the information in v becomes available (if τ < 0, it becomes becomes available immediately, at t = 0). In practice, we discretize continuous time t into 5 ms bins, corresponding to the 200 Hz low-pass filter frequency applied by Ding et al., so that there are 50 time steps between two syllable onsets.

All vectors w for the stimuli sequence of an experimental condition are concatenated into a single matrix W that captures the entire session’s time sequence, with a different random order of trials for each of the twelve simulated participants. This matrix has N rows and 50 columns per syllable of the stimulus sequence.

The values of the model’s three free parameters (μ, β, and σ) were chosen to obtain results on the English sentences that were visually similar to those of Ding et al., using a different set of word vectors (see S4 Appendix).

Analysis

Following Ding et al., we applied a Discrete Fourier Transform to obtain a power spectrum for each experimental condition and each simulated participant. The individual rows of matrix W, each representing the time course in a single dimension of word vector space, were transformed to the frequency domain. Next, these per-dimension power spectra were averaged over the N dimensions to obtain the power at each frequency bin. Frequency bin width was 1/9 Hz for Chinese and 1/11 Hz for English, as in Ding et al. Again following Ding et al., we tested whether the power at each frequency bin significantly exceeded the average of the previous and next two bins using one-tailed t-tests with false discovery rate correction [23]. Matlab code for computing, analyzing, and plotting power spectra is available online (S1 Code).