Terminology—Acceptability, Plausibility, Meaningfulness.

In the literature, various terms are being used for the concept of plausibility [28], and the term plausibility is used to describe different concepts [28], [9].

[28], [29] use the term plausibility (while emphasizing the difficulties in defining it), and state that it is often defined operationally: Plausibility is obtained through human ratings of plausibility. They also point out the apparently synonymous usage of other terms, like sensible and makes sense. In another study [30], those ratings are referred to as judgements of meaningfulness, without further defining this term. This term was also used in [7] to describe the relative acceptability of affix-word combinations. Conversely, [31] used the term semantic deviance to describe expressions that cannot be interpreted in normal communicative contexts and are therefore implausible.

In the model in [9], plausibility is given if a compound describes something that the listener can refer it to (for example, the compound eucalyptus bear is plausible if you know about the existence and eating habits of koalas). In this model, the acceptability of an interpretation for a compound is then a function of (amongst others) its plausibility.

For the remainder of this paper, we will assume as a working hypothesis that plausibility, acceptability, meaningfulness, and semantic deviance subtend the same latent variable. We therefore assume that these terms can be used interchangeably for our purposes. For the remainder of this article, we will keep to the term plausibility.

Stages of Plausibility Judgements.

As pointed out in [29], although plausibility ratings have often been used to explain various cognitive phenomena (for example in the areas of reasoning, memory, and problem solving), it received little attention as a variable of interest in itself.

To overcome this gap, these authors proposed the Plausibility Analysis Model (PAM) [32], [28], [29]. The main focus of this model are plausibility judgements for whole scenarios consisting of multiple sentences, such as The bottle fell off the shelf. The bottle smashed. However, it also provides a useful theoretical background for plausibility judgements on simpler expressions, such as noun compounds.

In this model, plausibility judgements are the result of two stages: A comprehension stage and an assessment stage. During the comprehension stage, a mental representation for the input (i.e., the compound) is obtained. The plausibility of this representation is then evaluated in the assessment stage. The main assumption in PAM is that it is assessed whether the obtained representation is in line with prior knowledge. Especially, it is examined whether the concepts that are part of the mental representation are coherent.

Linguistic Approaches—The Problem of Interpretation.

In the linguistic literature, the issue of how meanings are assigned to compounds, and to what extent these interpretations of a compound’s meaning can be predicted, for example from its constituents, is referred to as the problem of interpretation [2], [33].

In his seminal generative approach to compounds, [34] advocates the idea that compounds are transformations of sentences [35], or noun-like versions of sentences that are stripped of some grammatical elements and re-arranged. Consider as an example a compound such as stone wall. For the purpose of illustration, we will start from the sentence The wall is built out of stones. One possible transformation of this sentence is the sequence … wall built out of stones … which can be used in a noun-like fashion (e.g., The guardian continued his patrol on the wall built out of stones). The compound stone wall then is a transformation of this sequence, and can be used instead of the sequence: The guardian continued his patrol on the stone wall. The basic idea of this approach is that these examples share the same deep structure from which they are generated. The meaning of the compound is then given by the deep structure from which it was generated. The relation between compounds and syntactic structures is particularly evident for head-initial compounds in Romance languages [36], in which prepositional compounds are also observed [37]. In Italian, for example, the same compound can be expressed through a head-initial structure (e.g., cabina telefonica, phone booth, lit. booth telephone_adj) or a prepositional structure (e.g., cabina del telefono, lit. booth of the telephone_noun).

On the other hand, according the lexicalist approach to compounding [20], [38], [39], it is assumed that the lexicon and the lexical semantics of the constituents carry the workload of compounding, not the underlying deep structure. Thus, the lexicalist approach assumes that the constituents of a compound determine its meaning, and not its construction (see also [5]). This is illustrated in the Variable R Condition proposed in [20]: In the primary compound [X Y]_Z, the meaning of X fills any one of the feature slots of Y that can be appropriately filled by X.

The lexical semantic approach [39], [26] builds on and further specifies this point. According to Lieber [39], [26], the semantic representation of a morpheme (in this case, a constituent) consists of a semantic/grammatic skeleton that contains all its (semantic) features that are relevant to the syntax of a language. Examples in English are whether an entity is a concrete or an abstract noun, or whether it is static or dynamic. In addition to the skeleton, the representation also entails the semantic/pragmatic body, which includes other features of and knowledge about the constituent, for example that a dog has four legs and that it barks. The studies in [39], [26] then analyse compounding for the three classes of compounds [25] (we will focus on endocentric compounds here): For coordinate compounds such as singer-songwriter, that share a large amount of features, the skeleton and the body are assumed to be highly similar and therefore easily coindexed (coindexation in this context is to be understood as “identified as referring to the same entity”). They will also differ in some features, and those features can either be interpreted as being simultaneously true, as in the case of singer-songwriter, or mixed, as in the case of blue-green. For subordinate compounds such as taxi driver or football player, Lieber argues that the heads (driver and player) have free slots for arguments (specifying what is driven and what is played), and this role is filled by the modifiers. In most cases, such a process can work on the level of the semantic/grammatic skeletons alone. Finally, for attributive compounds such as horror story or doghouse, which are allegedly the most frequent and most productive in English [26], the case is somewhat different: Although their skeletons can be very similar (dog and house are both concrete objects), their bodies can differ quite substantially (a dog is animate, not human, has four legs and barks, while a house is not animate, and artefact, and has windows and a door).

In another approach, Jackendoff [40] proposes that interpreting the semantic structure of a compound relies on two factors: on the one hand, the head of the compound has to be identified, and on the other hand, the semantic relation between the constituents has to be determined. He identifies two main schemata for this semantic relation: One schema is the argument schema, where a compound [X Y] is an Y by/of/… Z. This schema is most prominently realized in subordinate compounds. Attributive compounds, however, can in most cases not be interpreted with this schema, and the relationship between the constituents—or, in other words, which features of the head are affected in which way by the modifier’s features—is not fixed and therefore free and potentially ambiguous, or promiscuous [40]: A dog house can be a house in which dogs live, or a house in the shape of a dog, or a strange house which consists of dogs as building blocks. Following Jackendoff, the modifier schema is applied in such cases: [X Y] is an Y such that some F is true for both X and Y. Interpreting the meaning of [X Y] then is identifying F, or, in other words, the specific relation between X and Y. Possible candidates for such a relation, which is argued not to be completely arbitrary but rather an element of a finite set of possible relations, include a LOCATION relation (Y is located at/in/on X, as for mountain pass), a SERVES AS relation (Y serves as X, as for buffer state), or a CAUSE relation (Y is caused by X, as for knife wound); for a more complete list of relations, see [40].

Taken together, the main idea of such lexical approaches is that both constituents are defined by a set of semantic features, which are combined, selected or changed in the compound generated from the constituents.

One commonality of many theories on compounding, including generative and lexicalist approaches, is the view that an important part of interpreting a compound’s meaning is to interpret the relation between its constituents, that is, to identify Allen’s [20] type R relation (e.g., [41], [5], [42], [40], [43], [26]). As an illustration, a wind mill is usually interpreted as a mill that is powered by wind, but other interpretations are also available given an appropriate context: for example, a wind mill could, in some other world, also be a mill that produces wind (compare flour mill). A major task of many of these theories is to identify possible relations between the constituents, and to classify given compounds with respect to these relations. For example, [43] postulates a set of nine different relations, which, amongst others, include a CAUSE relation (e.g., air pressure, accident weather), a HAVE relation (e.g., city wall, picture book), or a USE relation (e.g., wind mill).

Psychological Approaches—Conceptual Combination.

In the psychological literature, the process of combining two concepts into a new one (as for adjective-noun compounds or noun-noun compounds) is referred to as conceptual combination (see [15], [16] for reviews on this topic).

Probably the first psychological model of conceptual combination is the Selective Modification Model [44], [11]. This model assumes concepts to be stored in memory as prototype schemata, which consist of a set of dimensions. Each of these dimensions includes a range of features (the dimension colour, for example, can include the features red, blue and green), and each of those features is weighted by a numerical value of “votes” (for the concept sky, the feature blue probably has the highest vote count on the dimension colour, soon followed by grey). Furthermore, the model also postulates a numerical diagnosticity value to be assigned to the dimensions: For the concept sky, the dimension colour most likely has a higher diagnosticity than the smell dimension, while the opposite should be the case for perfume.

However, the focus of the Selective Modification Model were adjective-noun combinations, and not noun compounds. An early model dealing with noun compounds is the Concept Specialization Model [45], [46], [47], which can be considered an extension of the Selective Modification Model [16]. This model assumes a similar representation of concepts, namely as prototype schemata with slots (i.e., dimensions) and fillers (i.e., values on these dimensions). When a head noun is combined with a modifier, the concept given by the head noun is then altered as a function of the modifier concept. More specifically, it is assumed that the modifier fills in specific slots of the head noun concept, which is a specialization of the head noun concept. The selection and filling of slots is guided by background knowledge. In the case of the compound moon colonist, the head noun colonist might for example have a slot for LOCATION and for AGENT. When this concept is combined with the modifier moon, the LOCATION slot is then filled with moon. That moon is more suitable as a LOCATION than an AGENT is determined by the listener’s background knowledge on the nature of colonisation (usually, this is a process of people settling on some land), and of the moon (which is an area that could in principle be settled on). As can be seen, these approaches resemble the core idea of lexicalist approaches to compound meanings [20], [39], [26], which assume that one constituent of the compound (the modifier) specifies certain features of the other constituent (the head).

Over the following decades, several additional models on conceptual combination have been proposed [48], [49], [42], [12], [50], [9], [51]. As argued and illustrated in [16], those can be seen as extensions or specifications of the Selective Modification Model and the Concept Specialization model. Although they differ in their scope and theoretical assumptions on how the process of conceptual combination works, and how interpretations for compounds are obtained, they share the basic assumptions of concepts being represented as prototype schemata with dimensions. Furthermore, they assume that the combination process modifies the head noun’s values on these dimensions with respect to the modifier noun, which is an instantiation and specific implementation of identifying Allen’s (1978) Relation R.

Notably, the Competition Among Relations in Nominals (CARIN) model by Gagné [42], [52], [53] postulates that a crucial part of conceptual combination is to identify a thematic relation between the constituents of a compound (see also the present version of CARIN, the RICE model, for an updated formalization [54]). This approach is therefore very similar to linguistic theories that focus on relations between constituents to address the problem of interpretation ([5], [43], also see the respective paragraphs in the previous section). According to the CARIN model, relations are known from prior experience, and have to be filled in for a given compound that is encountered. Hence, the CARIN model assumes that a concept has slots for thematic relations that can link the concept to other concepts. The likelihood that a given relation is chosen for the interpretation of a given compound then depends on prior experience: For example, river mill will be most likely identified as a mill that is located nearby a river, since the modifier river if often used to establish a locative relation in compounds.

The Pragmatics of Conceptual Combination.

While most psychological models of conceptual combination are focussed on compositional semantics (i.e., how the meaning of the compound is formed as a function of its constituents), the Constraint Model [9] employs pragmatical principles of communication. Central to this model is the assumption that the speaker and the listener in a communicative situation are cooperative [55]. This especially implies that the speaker tries to choose the best-fitting expression in order to transfer an intended meaning to the listener.

From this assumption, [9] derive three pragmatical constraints concerning the meaning of compounds: As stated earlier, plausibility indicates whether the compounds refers to something that the listener can be assumed to know. If the listener does not know about the concept of koalas (and especially their eating habits), a more detailed description of the concept than eucalyptus bear would be more adequate. Diagnosticity indicates whether the combined concept is best identified by the specific constituents of the compounds. We can assume diagnosticity to be quite high for eucalyptus bear, which is surely more diagnostic of what a koala is than for example tree bear. Finally, informativeness indicates whether both constituents are actually needed (and sufficient) to identify the meaning of the combined concept. In the case of water lake, adding the modifier water is at best unnecessary, if not confusing in most contexts.

In the Constraint Model, the interpretation of a noun compound is then assumed to be the most acceptable one, while acceptability is a function of these three constraints. Note that acceptability here refers to the acceptability of different interpretations of a given compound, not to the acceptability of the compound itself. However, it seems reasonable to assume that the plausibility (in terms of meaningfulness, as discussed previously) of a compound is a function of the acceptability of its interpretation: A compound for which a good interpretation can be obtained should be considered more plausible than one for which even the best interpretation is not very acceptable.

Distributional Semantic Models.

In the theories of conceptual combination discussed so far, some major theoretical concepts remain underspecified. There remain free parameters, such as the dimensions and features a concept includes, and how exactly those are changed in a specific combination of a modifier and a head noun. Although models of conceptual combination have been successfully implemented computationally [11], [52], [9], these implementations rely on hand-crafted encoding of those parameters [56].

Distributional Semantic Models (DSMs) provide a possibility to address these issues. In DSMs, the meaning of a word is represented by a high-dimensional numerical vector that is derived automatically from large corpora of natural language ([57], [58], [59], for overviews on DSMs). For the remainder of this article, we assume that word meanings correspond to concepts ([60] provides a detailed discussion on this issue).

The core idea of distributional semantics is the distributional hypothesis, stating that words with similar meanings tend to occur in similar contexts [61]. This should also be reflected in the opposite direction: Words that appear in similar contexts should in general have more similar meanings than words appearing in different contexts. For example, the meanings of moon and sun can be considered to be similar as they often occur in the context of sky, sun, universe, light and shine.

By explicitly defining the notion of context, the distributional hypothesis can be quantified. The two most common approaches are to define context as the documents a word occurs in [62], [57], or as the words within a given window around the target term [63] (see [58], for the differences between these approaches).

We will illustrate the second option with a toy example. Assume we want to extract vector representations for the word moon. As relevant context words we take sky, night and shine, and we assume that two words are co-occurring if and only if they appear in adjacent positions in a sentence (technically, within an 1-word window). Scanning through the corpus, we then find 2 co-occurrences of moon and sky, 5 co-occurrences of moon and night, and 3 co-occurrences of moon and shine. Therefore, we can derive the following vector representation for moon:

The same procedure can be applied to other words as well. For example, counting co-occurrences between sun and sky, night, and shine might result in the vector

If the same context words (in the same order) and the same corpus were used to construct two word vectors, these will live in the same semantic space. In this case, it is possible to approximate how similar two word meanings are, usually by computing the cosine similarity between the two respective word vectors, which is defined as (1) for two n-dimensional vectors a and b. If there are only positive values in the vectors, as is the case for raw co-occurrence counts, the cosine similarity ranges between 0 (for orthogonal, that is unrelated vectors) and 1 (for identical vectors). In the example above, the cosine similarity between moon and sun is .71.

The vectors derived this way are typically further processed, by applying weighting schemes on the raw counts, as well as dimensionality reduction techniques [64], [65], [59]. The purpose of applying weighting schemes is to adjust for frequency effects: Usually, very frequent words (such as and or was) are less informative for the meaning of their surrounding words than infrequent words (such as cardiology or xylophone); furthermore, the similarity of two word vectors based on raw co-occurrence counts is considerably influenced by the words’ frequencies. The purpose of dimensionality reduction techniques, such as Singular Value Decomposition (SVD) or Non-negative Matrix Factorization(NMF), is to get rid of noise in the data, and to generate latent, underlying dimensions of meaning as context dimensions [57].

Distributional Semantics in Cognitive Science.

Originally, DSMs were designed as a method in computational linguistics and natural language processing, but soon became popular in cognitive science, mainly due to the success of popular models such as Latent Semantic Analysis (LSA; [62], [57]) or the Hyperspace Analogue to Language (HAL; [63]).

It has been shown in numerous studies that DSMs are a psychologically plausible approach to meaning [57], [66], [67], [68], [69], [70]. Apart from being able to account for various empirical behavioural phenomena, such as predicting human similarity ratings [57] or priming effects [67], [71], there are also more theoretical ways in which DSMs can be aligned with psychological theories: They can encode properties of concepts [69], [72], and provide an account of how we learn, structure and abstract from our experience and induce relations that were not explicitly stated or observed [57].

It is hereby more a contingent property rather than a defining feature of DSMs that they seem to be centred around word co-occurrences. This is mainly due to the availability of large text collections and the tools to process them, which are mostly practical issues. In fact, DSMs can also be designed to encode extra-linguistic information, which has already been done successfully with visual information [73], [74]. Therefore, DSMs should be seen as a formal description of how experiential input is organized and information is structured in our minds, by considering the contexts in which a stimulus (in this case, a word) was or was not present, and the contextual similarity to other stimuli. Indeed, even when considering purely textual input, the view that DSMs can only capture textual similarity is somewhat misguided: Studies by Louwerse [75], [76] show that DSMs do not only encode linguistic information, but also world knowledge and even information that is usually considered to be embodied, such as spatial-numerical associations [77]. As an example for the encoding of world knowledge, [75] show that lexical similarities between city names in LSA correspond to the actual geographical distances between those cities. The observation that language encodes a lot of information about the actual world is highly plausible given that, in many cases, language is used to talk about the world.

Furthermore, an important point concerning the two possible representations of word meanings (or concepts) as high-dimensional numerical vectors (as in DSMs) and as lists of features (as assumed in models of conceptual combination) has been made in [66] (compare [57], for an earlier version of this idea). They show that there is actually a correspondence between those two representations, as a vector representation can be seen as a probability distribution over different semantic topics (see also [69]). Therefore, the dimensions which constitute the vectors in DSMs can be interpreted as semantic dimensions of the respective words, or concepts [57], although it might be difficult to name those dimensions on an individual basis. In conclusion, vector representations of meanings DSMs are not just to be seen as refined co-occurrence counts, and DSMs should not be taken as inventories purely encoding lexical statistics.

Composition in Distributional Semantics.

At this point, we only discussed how meanings of single words are represented in DSMs. However, meanings can clearly also be assigned to more complex expressions, and models of meaning should account for that. Especially, it is important to be able to obtain meanings also for novel expressions that were not encountered before, since the possibility to generate novel combinations is an essential property of language.

Recently, the topic of compositionality in DSMs has received considerable attention [78], [79], [80], [81], [82]. The basic feature of compositional DSMs is that the vector representation of a noun compound lives in the same semantic space as the vector representations for single words, and it can be computed arithmetically on the basis of the elements in the expression (see the Methods section for technical details). In the case of noun compounds, the compound vector is therefore based on the modifier noun and the head noun. Importantly, such vectors can also be computed for compounds that were never attested in a corpus.

In general, the relation between the compound meaning and its constituents can be stated as (2) with c being the vector representation of the compound, m and h being some representation of the modifier and the head (not necessarily in vector terms, see Methods), and f being a function linking those representations. Note that this formulation is identical to other linguistic theories of compound meanings, for which a main objective is to identify the function f for a given compound [40].

This relation implies that each dimensional value p_i of vector p is itself dependent on the modifier and the head noun of the compound. Therefore, compositional models in DSMs are comparable to psychological theories of conceptual combination, which also assume that the dimensional values of the combined concepts are a function of the compound’s head and modifier (as described earlier).

In this perspective, we can see compositional methods for DSMs as an algorithmical formalization of conceptual combination: Instead of hand-crafted feature lists, concepts are represented as data-driven, high-dimensional numerical vectors; and the process of combination itself is formalized by applying arithmetical operations, resulting in a vector representation for the compound.

In summary, we assume that the product of the comprehension stage for a compound is as a vector, derived compositionally on the basis of the compound constituents. Following [66], this vector representation corresponds to sets of features of the combined concept.

Plausibility Measures in Distributional Semantics.

In the study in [31], several measures in distributional semantics for phrase plausibility were employed (also called semantic transparency measures). It has already been shown in other studies that such measures are useful in predicting the plausibility of adjective-noun phrases [83], [31] as well as word-affix combinations (such as re-browse vs re-wonder) [7], and resolving syntactic ambiguities for three-word compounds [84]. In this section, we will describe those measures and the rationale behind them

• Head Proximity. Head Proximity is defined as the cosine similarity between the expression in question and its head (in our case, between the noun compound and its head noun), so (3) with c being the phrase in question, and h being the vector of the head noun. Hence, the head proximity indicates how related a compound meaning is to the meaning of its head noun, or how much this head noun meaning contributes to the compound meaning. In that, Head Proximity is related to the concept of analysability in linguistic theories of compounding [85], [86], which is defined as “the extent to which speakers are cognizant (at some level of processing) of the contribution that individual component structures make to the composite whole” [86] (p. 457). It has been argued that analysability is a gradual phenomenon and therefore a continuum rather than a binary notion; this is in line with our approach, which defines Head Proximity as a gradual cosine similarity.
  The general idea here is that a higher head proximity indicates a more plausible phrase. For example, if a house boat is still highly related to the concept of boat, one would expect house boat to be a rather plausible phrase.
  As discussed earlier, this assumption is in line with conceptual coherence, as an indicator of how well a combined concept can be fitted to prior experience with the respective head concept. Following the constraint of diagnosticity [9], combined concepts should be some instance of the category described by the head noun, or at least share a sufficient amount of features with it. Otherwise, the usage of another head noun to create the compound would have been a better choice.
• Modifier Proximity. The same notion of proximity between a phrase and constituent can also be applied to the modifier: (4) with p being the phrase in question, and m being the vector of the modifier noun. The rationale of diagnosticity, as already discussed for Head Proximity, can be applied here: In order for a phrase like house boat to be plausible, it should also be related to the concept house, because there should be a reason that exactly this modifier is included in the phrase. Therefore, the concept should be analysable with respect to the modifier, that is the modifier’s contribution to the compound meaning should be identifiable.
  So far, we have argued that, according to the diagnosticity constraint, higher proximities between the constituents and the phrase should result in more plausible phrases. However, according to [9], the influence of diagnosticity is modulated by informativeness, that is whether both constituents are necessary and sufficient to constitute the intended compound meaning. Therefore, the relation between the proximities and plausibility might not be a linear one, or maybe not even monotonously positive. For example, it can be argued that in the case of rather non-informative compounds such as water lake, too close a relatedness between the constituent meanings and the compound meaning leads to relatively lower plausibility judgements.
• Constituent Similarity. Constituent Similarity is defined as the similarity between its modifier noun and its head noun: (5) with m being the vector for the modifier, and h being the vector of the head noun. [56] found the LSA cosine similarity between the two constituents of a phrase to be predictive for its plausibility: This similarity was larger for typical adjective-noun pairs (such as sharp saw) than for atypical adjective-noun pairs (such as mortal god), and this similarity again was larger than for noun compounds. These differences correspond to differences in the ease of comprehension for these compound types, as indicated by human ratings, lexical decision reaction times, and classifications whether a compound is plausible or not [47].
  However, note that Constituent Similarity captures conceptual coherence only on the level of single word meanings: If the two concepts that are combined are coherent, the compound should be perceived to be more plausible as when they are incoherent. However, if the plausibility of a compound was only determined by the similarity between its constituents, it would be possible to judge it without having a representation for the compound meaning. This is hard to bring in line with the literature on conceptual combination.
• Neighbourhood Density. For each vector living in the semantic space, its m nearest neighbours are defined as those words having the highest cosine similarity with the said vector. Neighbourhood Density refers to the average similarity between a vector and these neighbours (6) with c being the (compound) vector in question, k being a fixed number of nearest neighbours to be considered, and n_i being the ith nearest neighbour to p.
  The idea behind selecting neighbourhood density as a measure for plausibility is the assumption that plausible expressions should live in a higher-density neighbourhood than implausible ones. The meaning of a more plausible expression should be quite similar to other, already known concepts, and it should be quite clear from that neighbourhood which meaning the expression conveys. A less plausible expression, on the other hand, should be fairly isolated from other concepts, which makes it hard to tell what it means.
  Since neighbourhood density is a measure of how similar a concept is to various already known concepts, it is in line with the notion of conceptual coherence as a determinant of plausibility.
• Entropy. Entropy is a prominent concept in information theory, indicating how far a (probability) distribution deviates from a uniform distribution. For an n-dimensional vector p with a value of p_i on the ith dimension, it is defined as (7)
  High values of entropy indicate a distribution that is close to a uniform distribution, while lower values indicate a more diverse distribution, with peaks in some dimensions and very low values in others.
  Entropy can be hypothesized to predict the plausibility of an expression from its vector: A vector for a plausible expression should have high values on the dimensions that are highly diagnostic for the concept, and low values on other, irrelevant dimensions. Following [66], such a vector represents a concept that has defined features. On the other hand, a vector that is very close to a uniform distribution has no specific dimensions with which the respective concept is likely to occur. Therefore, such a concept has no distinct features, and should therefore be implausible.

Outlines for the Present Study.

In this study, we want to investigate which factors determine the plausibility of noun compounds. To achieve this, we employ compositional methods in distributional semantics in order to obtain formalized vector representations for these compounds, and use different plausibility measures that capture different aspects of conceptual coherence in compounds.

In this, our study has a similar approach as the study in [31]. However, we extend this study in several respects: First, we focus on noun compounds instead of adjective-noun phrases and therefore to another class of expressions and conceptual combinations. While most literature on conceptual combination accounts for both cases [16], some models, such as the Selective Modification Model [44], [11] cannot account for noun compounds, as discussed earlier.

Secondly, while [31] have concentrated on plausibility judgements only for unattested and hence novel adjective-noun phrases (such as spectacular sauce), we want to investigate attested as well as novel noun compounds. This will provide us with a more comprehensive and general picture of what influences plausibility judgements, for a variety of differently familiar compounds.

Finally, the focus of the study in [31] was to find out which compositional method in combination with which plausibility measure predicted human plausibility ratings best. This approach gives computationally efficient results, but does not take into account whether different measures play differently prominent roles in judging plausibility. Furthermore, potential interactions between the measures are neglected. Such interactions are suggested in [9], by assuming that diagnosticity and informativeness should modulate each other. In our study, instead of choosing the single best predictor, our aim is to model plausibility judgements for noun compounds with the best-fitting combination of plausibility measures, including possible non-linear effects and interactions.

Corpus.

The corpus used to derive the semantic space resulted from the concatenation of three corpora: The British National Corpus (http://www.natcorp.ox.ac.uk/), the ukWaC corpus obtained from web sources (http://wacky.sslmit.unibo.it/) and a 2009 English Wikipedia dump (http://en.wikipedia.org). This corpus contains a total of about 2.8 billion tokens—an amount that is comparable to a lifetime’s total language experience (which is which is estimated to be about 2.2 billion words; [88], [89]). The corpus has been tokenized, lemmatized, and part-of-speech tagged using TreeTagger [90] and dependency-parsed using MaltParser (http://www.maltparser.org).

We only considered the lemmatized version of each token in our analysis (i.e., different word forms of monkey, such as monkey and monkeys, will both be mapped onto the lemma monkey). For a discussion on lemmatization, see [59]. In the remainder of this section section, we refer to those lemmata when we speak of words.

Vocabulary.

In a semantic space, each row gives the vector representation for a word. Word vectors were computed for the following words:

• The 20,000 most frequent content words (nouns, verbs, adjectives, adverbs) in our source corpus.
• The constituents of the word pairs in the data set from [30]
• All the words that were part of any training set for the composition methods we employed (see the section on Composition Methods and S1 Appendix for details).

In total, this resulted in 27,090 words populating the semantic space (i.e. 27,090 row vectors).

Constructing the Semantic Space.

The context dimensions (i.e., the columns of the semantic space) were set to be the 20,000 most frequent content lemmata (nouns, verbs, adjectives, adverbs) in the source corpus. Therefore, the initial semantic space is a 27,090 × 20,000 matrix.

The cells of this semantic space were filled up by sliding a ±2-word context window over the corpus [63]. Each word in the vocabulary was therefore considered to co-occur with the two context words preceding and following it. For each co-occurrence of vocabulary word i with context word j, the value in cell (i, j) of the semantic space was increased by 1. Only co-occurrences within sentences were counted. The procedure results in a raw count matrix.

In a next step, a positive Pointwise Mutual Information (PMI) weighting [91] was applied to this raw count matrix. The PMI measure is a widely used word association measure and defined as follows: (8) with a and b being two words, p(a, b) being their probability of co-occurrence, and p(a) and p(b) being their marginal probability of occurrence. PMI therefore measures whether the actual co-occurrence probability of two words is higher than their probability of randomly co-occurring. Positive PMI (PPMI) is a variation of this measure where resulting negative PMI values are set to zero. It has been shown that applying PPMI weightings to the raw counts considerably improved the performance of DSMs [64].

In a last step, Non-Negative Matrix Factorization (NMF) [92] was used to reduce the dimensionality of the weighted count matrix. Dimensionality reduction techniques, especially Singular Value Decomposition (SVD), are used very often in DSMs, and improve their performance considerably [57], [93], [65]. We decided to use NMF instead of SVD, as it was shown to give better empirical results [92]. Furthermore, it has been shown that employing Non-negative Matrix Factorization (NMF) as a dimensionality reduction technique on window-based semantic spaces produces dimensions that can also be interpreted in a probabilistic fashion as a distribution over different topics or features [94], as is the case for topic models [66]. We also performed the computations reported here using SVD, which gave very similar results. NMF is similar to SVD, with the difference that all resulting vectors only contain non-negative values (which is not necessarily true for SVD). The algorithm was set to reduce the weighted count matrix to a semantic space with 300 dimensions, based on previous findings [57].

The free software toolkit DISSECT [95] was used to perform the computations needed to construct the semantic space.

Training the Lexical Functions.

The training set for the Modifier Lexical Function consisted of all the noun pairs in the corpus (a) where the first noun appeared as a constituent in the item set (and hence as a modifier, in the attested or the reversed order), and (b) that occurred at least 20 times in the corpus. There are 391 different modifiers in the item set. Since estimations are unreliable if there are not enough training items for a specific modifier, we removed 163 modifiers for which there are less than 50 different training pairs in our source corpus. For the remaining 228 modifiers, a total of 52,351 training pairs were found, with up to 1,651 different training pairs per modifier noun. Pairs that were part of the data set were not used as training items.

The lexical function matrices were estimated and compound vectors were computed using DISSECT [95].

Since we eliminated 163 modifiers from the data set, we obtained 1,699 compound vectors (881 for attested and 818 for unattested compounds).

Plausibility Measures.

As variables for predicting the plausibility of the compounds, we employed Neighbourhood Density (setting the size of the neighbourhood to k = 20 without tuning) and Entropy, computed on the 1,699 compound vectors that we derived compositionally. Head Proximity and Modifier Proximity were also computed on these compound vectors, with the vector representations for the head noun (or modifier noun, respectively) obtained from our semantic space. Furthermore, we computed the Constituent Similarity between modifier noun and head noun from their vector representations in our semantic space.

Covariates.

In addition to the plausibility measures, we considered several linguistic covariates:

• Length (in letters) for modifier and head nouns
• Logarithmic frequency of modifiers, heads, as well as the modifier-head pairs in both orders according to the 201-million-word SUBTLEX corpus [96]. We avoid the term compound frequency and use modifier-head pair frequency in this article, since every occurrence of modifier and head next to each other, not necessarily as a compound, is counted for this frequency. Thus, for the compound tree apple, we considered the logarithmic frequency of both tree apple as well as apple tree as a covariate. To deal with zero frequency words and bigrams, we used the Laplace transformation for frequencies [97].
• Family size for modifiers and heads, according to our source corpus. Family size specifies in how many different compounds a modifier noun is used as modifier, or a head noun is used as head
• Pointwise Mutual Information between the modifier noun and the head noun [91]. This variable specifies how the probability of two nouns actually occurring together relates to the probability that they randomly occur together, and is a measure for the association between two words.

Head Proximity and Modifier Proximity.

As can be seen in the upper left panel of Fig 1, Head Proximity has a positive effect on the plausibility of compounds: The higher the Head Proximity is, the higher plausibility ratings tend to be. Since this statement holds for all levels of Modifier Proximity, this is a general positive effect of Head Proximity.

Considering that the role of the head noun in a compound is to define the semantic category the compound belongs to [19], this effect can be explained as an effect of the ease of categorization. In general, compounds are rated as more plausible the closer the respective combined concept is to the category (or concept) denoted by the head noun, that is the easier it is to interpret them as an instance of this category. This is in line with the common finding that the relatedness to a category prototype is a major determinant of whether a specific concept is a member of that category [103]. As discussed previously, distributional semantics leads to representations of concepts that can be interpreted as prototype schemata. Note that, in such an interpretation of our results, the view that the compound is a hyponym of the head and therefore a member of the head category is very prominent. This is not strictly speaking logically true for all compounds, since there exist exocentric compounds such as metalhead (but see [27], [104] for critical views on the topic of exocentricity). However, this does not imply that our analysis is restricted to endocentric compounds only. Instead, we assumed as a working hypothesis in the present study that human judge apply the same mechanisms for judging the plausibility of noun compounds of different categories. The empirical validity of this working hypothesis has to be sorted out in future research.

Examples for compounds with low and high Head Proximity values can be seen in Table 2. As can be seen from these examples, it is much easier to identify the compounds with high Head Proximities as members of the head noun category, while the same is very hard (or almost impossible) for compounds with low Head Proximities.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Example items for compounds with low vs. high Head Proximity Values.

https://doi.org/10.1371/journal.pone.0163200.t002

However, this effect of Head Proximity is strongly modulated by the Modifier Proximity. This interaction emerges in two patterns (see the upper left Fig 1). First, the effect of Head Proximity is steeper if the Modifier Proximity is medium-high, so already small raises of Head Proximity come with higher plausibility ratings. Stated in other terms, plausibility ratings drop off if the Modifier Proximity gets too high or too low, in comparison to medium-high Modifier Proximities (except for very high Head Proximities). The notion of informativeness [9] can be applied to explain this effect: If the meaning of a modifier is too distant from the compound meaning, it is hard to understand how exactly the modifier contributes to the compound. This difficulty comes with relatively low plausibility ratings. If, on the other hand, the modifier is too closely related to the compound, it can be considered as redundant, and there is no justification to include it in the compound at all. This redundancy violates the assumption that compounds should be informative, which comes with lower plausibility ratings.

That redundancy has negative effects on the interpretability of noun compounds has already been noted in [5], who specifies three conditions that cause redundancy: The modifier and the head noun refer to the same set of entities (e.g., lad boy); the set of entities referred to by one constituent is a proper subset of the set referred to by the other constituent (e.g., horse animal); or every instance of the head category is necessarily or typically an instance of the category denoted by the compound (e.g., water lake).

Note that, in our study, the representations for the compounds were derived compositionally from their constituents. In that light, Head Proximity and Modifier Proximity can be seen as a proxy of the contribution of the head noun and modifier noun to the combined concept: A high Head Proximity indicates that the meaning of head noun contributes highly to the compound meaning, as does a high Modifier Proximity with respect to the modifier (those two are not mutually exclusive, it can be the case that both constituents contribute highly or almost nothing to the combined concept). Therefore, our results indicate that redundancies occur when the contribution of the modifier noun, but not the head noun is too high in the combination procedure.

This point can be illustrated with some example items, see Table 3. As can be seen, items with an “optimal” medium Modifier Proximity appear to be intuitively plausible. On the other hand, for items with a low Modifier Proximity, the contribution of the modifier to the compound is not clear at all; and items with a high Modifier Proximity appear to be highly redundant.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Example items for compounds with different Modifier Proximity values, all with medium-high Head Proximity values (between .3 and .5).

https://doi.org/10.1371/journal.pone.0163200.t003

However, for compounds with a high Head Proximity value, while the drop-off in plausibility for low Modifier Proximities is still present, the effect for high Modifier Proximities is different: For these items where both Head and Modifier Proximity are high, the model predicts very high plausibility ratings. This effect might truly be one of a specific interaction between Head Proximity and Modifier Proximity, in that high values on both do not invoke the informativeness issues discussed before. More specifically, once the Head Proximity reaches a certain threshold (of about.65 in our data), the drop-off for high Modifier Proximities no longer appears. In those cases the high Head Proximity could just override those issues, since the compound is very easy to interpret as an instance of the head category, which might be more important than having an informative phrase ([9] also postulate that informativeness plays a subordinate role compared to the constraints of plausibility and diagnosticity).

Upon inspecting these items, however, we find a relatively large amount of lexicalized compounds: rain cloud, swimming pool, cheese cake, chicken salad and river valley are amongst them. We therefore propose to be cautious with regards to the generic interpretation of this effect, since it might be driven by other factors such as lexicalization.

Constituent Similarity and Modifier Proximity.

The upper right panel of Fig 1 shows the second interaction effect, between Constituent Similarity and Modifier Proximity. This effects consists of two main components: We find no effect for Constituent Similarity if the Modifier Proximity is above a certain threshold (about.4). Below that threshold, we find a positive effect for Constituent Similarity. For most items, this effect only predicts a very small gain in plausibility, although it is little bit higher if the Modifier Proximity is very low. Note that, although the model predicts drop-offs in plausibility for highly similar constituents, there are no data points after these drop-offs the model could be fitted on. Therefore, these drop-offs are most likely artefacts caused by the smoothing techniques used to estimate the model.

The small positive effect of Constituent Similarity is in line with the findings of [56] that more similar constituents predict more plausible compounds. However, as indicated by our analysis, this is not the case for all compounds, since this effect is absent if the Modifier Proximity exceeds a certain threshold (it should be noted here that [56] also conclude in there study that there is more to conceptual combination than just the similarity between constituents). We propose two explanations for this interaction:

The first possibility is that Constituent Similarity information is only used when the Modifier Proximity is low, that is when it is not clear how the modifier meaning contributes to the compound meaning. Such an interpretation assumes a positive effect of Constituent Similarity, but only for low Modifier Proximities. In that case, Constituent Similarity might help in overcoming interpretation difficulties that are caused by the opaqueness of the compound with regards to the modifier. If on the other hand the modifier’s contribution to the phrase meaning is sufficiently clear, there is no need to use this information, since the compound is already interpretable enough, and there is no need to consider Constituent Similarity.

Example items with low vs. high values on Modifier Proximity and Constituent Similarity that are in line with this interpretation can be found in the upper four cells of Table 4. For items with high Modifier Proximity values, such as baby rabbit, it is intuitively clear how the modifier contributes to the compound meaning, and therefore no further information on the similarity between modifier and head noun needs to be considered. On the other hand, for items with low Modifier Proximity values it might not be completely obvious how the modifier contributes to the compound meaning (is a pie salmon a rather round kind of salmon, or a salmon filled with something, or a salmon to be put in a pie?), but the general similarity between the constituents (both are some kind of food) makes it easier to align and combine them into a single concept.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Example items for compounds with different low vs. high Modifier Proximity values, crossed with low vs. high Constituent Similarity values.

https://doi.org/10.1371/journal.pone.0163200.t004

The second possibility to explain the interaction again considers the notion of informativeness, similar to our interpretation of the first interaction. Under this interpretation, we assume that Constituent Similarity generally has a positive effect on plausibility, but this effect is overshadowed by redundancies that occur when Modifier Proximity exceeds a certain threshold. In this case, the generally positive effect of Constituent Similarity and the negative effect caused by redundancies cancel each other out, and therefore we do not find a positive effect. Therefore, this second interpretation assumes a negative effect of high Modifier Proximity values that counteracts a positive effect of Constituent Similarity. Examples for this explanation can also be seen in Table 4, in the lower part of the bottom right cell, and include cases such as child infant. Of course, the similarity between child and infant is obvious, but the modifier child does not provide any semantic contribution to the compound, over and above the one brought upon by the head infant.

However, it is surely possible that both of the proposed mechanisms play a role in our study, and contribute to the pattern of results we found.

Constituent Similarity and Pair Frequency.

The third interaction, between Constituent Similarity and the frequency of the modifier-head pair, is shown in the lower left panel of Fig 1. As can be seen there, the pair frequency has a positive effect on the compound plausibility; however, this effect becomes smaller the more related the constituents are to one another.

It is a common finding that frequency (i.e., familiarity) has a positive effect on plausibility of noun compounds [105], [30]. Our results extend these findings, as we find that this effect is modulated by the similarity between the head and the modifier (without considering Constituent Similarity, our model would also have identified a positive main effect for this frequency, see S2 Appendix).

We explain this effect analogously to the first explanation offered in the previous section: Information about frequency is used more as the compound becomes less coherent, in terms of the similarity of its constituents. This might indicate that humans draw back to the very basic property of familiarity if it difficult to see how the constituents of the compound relate to one another. However, note that the model does not actually predict lower plausibility ratings for highly frequent items with high Constituent Similarities, but only a smaller boost in plausibility as compared to items with low Constituent Similarities.

Similarly to the previous sections, we present some item examples for this effect in Table 5. The examples with high Constituent Similarities but low frequencies such as door cabin show that, while the constituents are clearly somehow related to one another, the fact that those compounds are virtually never used results in a “strangeness” makes it hard to judge them as being plausible.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Example items for compounds with different low vs. high Constituent Similarity values, crossed with low vs. high frequency values.

https://doi.org/10.1371/journal.pone.0163200.t005

Furthermore, considering the high-frequency items, it is clear on an intuitive level that items from both groups are frequently used. Note also that the first group contains some idiomatic compounds (such as rock star and sea lion) for which the relation between the constituents is not very clear without knowing what the compound describes. To interpret those compounds, readers might therefore heavily rely on the familiarity with the compound to judge its plausibility. For compounds such as chocolate cake, on the other hand, the relation between the constituents is quite obvious, and there is no need to rely on stored knowledge about the combined concept to interpret them.

Another possible explanation for the negative relation between Constituent Similarity and plausibility of the compounds could be the claim in [5] that too similar constituents could result in implausible compounds. However, [5] explicitly refers to highly similar, but mutually exclusive constituents, such as butler maid or husband wife. Upon inspecting the items with high Constituent Similarities, we did not find such items (except for—maybe—tea coffee and coffee tea, with a Constituent Similarity of .86). Therefore, this explanation does not hold for our results.

A Mechanism for Plausibility Judgements.

Taken together, our result suggest a mechanism such as follows that guides judgements on the plausibility of noun compounds: Initially, a representation for the meaning of the compound is constructed compositionally. As this representation is obtained, it is checked to what extent the meaning of the head noun and the meaning of the modifier contribute to the meaning of the compound. If the contribution of the head noun is clear, and the contribution of the modifier is clear but not redundant (or if both the contribution of the head and the modifier are very clear), the compound is judged as relatively plausible. If however the contribution of the modifier is not clear, the similarity between the head and the modifier is used as a back-up resource. In this case, a higher similarity can help to make an otherwise difficult compound more plausible. Another possibility is that a higher similarity generally comes with a higher plausibility, but this positive effect can be “blocked” by redundant modifiers. Furthermore, if this similarity between the modifier and the head is low, the familiarity with the compound can be exploited as an additional back-up plan, with a higher familiarity boosting the perceived plausibility of compounds with dissimilar constituents.

Thus, our results suggests a threefold mechanism, with each step involving different levels of semantic complexity: In the first part, a meaning representation for the whole compound is necessary; the second part operates on the level of single words (the constituents); and the third part operates on the level of familiarity, which does not necessarily imply any representation of meaning. Therefore, in this proposed mechanism, in order to judge the plausibility of compounds humans rely on lower-level information as back-up mechanisms in cases where higher-level information is not clear or redundant.

Importantly, with the account presented here we do not claim that humans go through any of these processes consciously, in that they explicitly judge how well they can relate a tree house to a house, or in which way and to what extent the meaning of tree contributes to the meaning of tree house, in order to produce plausibility judgements. We rather propose, following the outlines of the Plausibility Analysis Model [29], that they initially try to build up a representation for tree house, and then judge whether this representation they built makes sense, and how well it fits their prior experience. The measures we proposed, and that we found to predict plausibility judgements, pick out on different aspects of such an alignment: How well does the combined concept fit into the relevant category (does it make sense for some house to be a tree house, or a neck house)?; can the role of the modifier and its contribution to the compound be identified (what exactly is an eggplant house, and how does a roof house differ from any normal house)?; and can the constituents be aligned (what do beard and house have in common to be combined into beard house)? Therefore, we argue that the measures we propose are aspects of a noun compound that (partly) constitute its plausibility. They can be used to describe whether a person is able to arrive at a representation she considers meaningful and plausible, or, possibly more importantly, ways in which she fails to do so, leaving her with a “fuzzy” representation she cannot integrate. Whether humans actually use this information in terms of cognitive processes, as in the model proposed in this section, remains subject to further empirical investigations.

Note that our argumentation along the lines of the Plausibility Analysis Model [29] implies a temporal sequence in plausibility judgements, in that we assume that a representation for the compounds is formed before the plausibility of this representation is assessed. This assumption is in itself an empirical hypothesis, which could be tested in studies employing compositional vs. non-compositional items (i.e., semantically transparent vs. opaque items), as well as items with high vs. low plausibility judgements. These could be separate classes of items, or the factors could be crossed in a 2 × 2 design. This material can then be presented to participants in an eye-tracking paradigm [107], with tasks such as just reading the compounds, lexical decision tasks, or timed plausibility judgements. Our hypothesis for such a study would be that differences between compositional vs. non-compositional items should be observed already in the first fixation duration [106], [108], while differences between plausible and non-plausible items should only emerge at later stages, and therefore only at the total gaze duration. A similar approach using timed plausibility judgements could also be implemented in an ERP paradigm, where differences between compositional vs. non-compositional items should be observed at earlier components of the ERP than differences between plausible vs. non-plausible compounds (compare [109] for such an analysis of early vs. late processing differences in ERPs).

Explicit Judgements vs. Processing.

Note that we do not necessarily assume that these parts have to take place in the sequential (or even hierarchical) order they were described in. In principle, we can also assume that the information is processed in a massively parallel and interacting fashion. In such a model, the different types of relatedness, as well as the information on familiarity, are checked at the same time, and then integrated in a final step to come up with a plausibility judgement. The data we considered includes explicit plausibility judgements, and no timing information. Therefore, while we can make statements about which factors influence those plausibility judgements, we cannot make claims about the temporal order in which these factors are considered, and about the processing of noun compounds. Indeed, a parallel explanation of our results would be in line with most recent models of compound processing [110], [111].

In a study on the processing of noun compounds, [112] investigated reaction times in lexical decision tasks from the English Lexicon Project [113]. In that task, participants had to decide whether a compound presented to them existed as a word or not (the compounds where therefore written as one string, for example swordfish and not sword fish). [112] found that those lexical decision times could be predicted from the relatedess between modifier and compound as well as between head and compound, according to DSMs (the measures we refer to as Modifier Proximity and Head Proximity): they were relatively short if both were high, and longer if any one of them was low. Results in line with these findings were also obtained in [114], wo found that compounds with an transparent head took shorter time to process than compounds with a non-transparent head. This pattern of findings resembles parts of our results in this study, where we find high plausibility ratings for compounds where both Head Proximity and Modifier Proximity are high. However, different to our results, [112] did not find shorter reaction times also for lower Head Proximities if the Modifier Proximity was medium-high.

Thus, we find relatively different results for explicit judgements on the plausibility of noun compounds as compared to the processing of noun compounds. Therefore, we assume that two at least partly different mechanisms are involved in the two different tasks. While it is necessary for explicit judgements on the plausibility to have an interpretable representation of the compound meaning that can be elaborated on, lexical decisions do not depend on such a representation. In the latter, only the existence of the compound as a lexicalized word has to be judged, and not whether it is plausible, which can also be the case for novel compounds. Also, lexical decision usually implies a certain degree of time pressure to the participants; as a consequence, the semantic access in this task may be coarser, with no role to be played by the nuanced interplays emerging in the present study.

Compositionality.

Crucially, as stated above, we assume that the meanings of compounds are derived compositionally, that is, they are constructed from the constituents when a compound is encountered. Our results show that we can predict human plausibility ratings of compounds by applying plausibility measures on distributional vectors that were derived using compositional methods in distributional semantics. Since a large part of the data set we used consisted of non-attested, that is novel noun compounds, we had to derive meaning representations also for those novel compounds. To accomplish this, compositional methods are necessary, since we cannot derive a representation for those compounds on the basis of prior experience with them (i.e., their co-occurrences the corpus). Compositional methods in DSMs provide a powerful way to derive such meaning representations for novel compounds, since these can be obtained for any compound where the constituents are known (more specifically, for the Lexical Function method we applied, the head noun must be known, and the modifier must have been encountered as a modifier in compounds before).

However, the utility of compositional methods in DSMs is not purely practical. Indeed, it has been proposed that these mechanisms may constitute plausible models for meaning combination at the cognitive level. In fact, in the present paper the same compositional method was able to successfully predict plausibility judgements for familiar, attested compounds. That is, our results are based on representations for those familiar compounds that also are obtained compositionally. In that, we are in line with the view that composition (i.e., conceptual combination) always takes place in the interpretation of compounds, for novel compounds as well as for familiar, more lexicalized ones [115], [116], [117]. According to [115], while it is obvious that the meaning of novel compounds has to be derived compositionally, as they cannot be represented in the mental lexicon, there are also reasons to assume that the meaning of familiar compounds is obtained compositionally as well. For example, [116] found that reaction times in sense-nonsense judgement tasks (as well as lexical decision times) for familiar compounds such as snowball were faster if they were primed with another compound that used the same relation between modifier and noun (such as snowfort, with a MADE OF relation) as opposed to a compound using a different relation (such as snowshovel, with a USED FOR relation). The same pattern can be found for novel compounds as target words [53]. These results show that even highly lexicalized compounds such as snowball are not represented as a fixed entry in the mental lexicon, but the relation between the constituents is represented, and therefore the constituents themselves are also represented. This in turn suggests that conceptual combination always is involved in obtaining representations, even for highly lexicalized compounds. More arguments for this claim can be found in [115]. Furthermore, it has been shown that conceptual combination is a very fast process [108], [7]. This suggests that conceptual combination is a rather automated process, which takes place whenever a compound is processed.

Implications of the Lexical Function Method.

The specific method we applied, and that gave the best initial results, was the Lexical Function method [81], [82], in which the modifier is conceptualized as a function that is applied on the vector representation of the head noun. In that, the modifier and the head play two very different roles in the composition process: The head noun is represented as a vector in the semantic space, with a number of dimensions and numerical values on these dimensions (which can be translated as features, Griffiths et al., 2007, Dinu & Lapata, 2010). Therefore, as in the literature on conceptual combination, the entity described by head noun is represented as a prototype concept. The role of the modifier is then to change these numerical values on the dimensions. More specifically, the matrix representation for the modifier as a Lexical Function gives an exact instruction of to what extent each dimensional value of the head noun vector influences each dimensional value of the compound vector. In that, the modifier guides the composition, in that it specifies how exactly the compound representation is obtained from the head noun representation, given a specific modifier. In more general terms, the head noun provides a structure for the composition process, in terms of a concept representation, and the modifier provides the process to be applied on that structure. This is in line with various views on conceptual combination [42], [52], [15], [16], and can, in our view, also be seen as an abstraction of lexicalist approaches to compounding (e.g., [39], [26]), with regard to the underlying idea that the meaning of a compound is obtained from its constituents, with features of the head noun being changed with respect to the modifier.

On a cognitive level, the Lexical Function composition method can be interpreted in terms of association learning [7]: Since in DSMs vector representations of word meanings are derived through co-occurrence counts, these representations are obtained through learning which words occur together, and which words occur in the same contexts (see also [57]), for example by association. Lexical Functions are learned associations on a higher level: A Lexical Function representation for a given modifier is obtained by learning from compounds including this modifier, and it stores the information on how the (distributional) meanings of the compounds are related to the meanings their respective head nouns. For example, the Lexical Function for moon captures the relation between walk and moon walk, landing and moon landing, stone and moon stone, and so on. It has thereby learned the regularities in the relation between a variety of compound meanings and head noun meanings that are specific to the modifier moon. When encountering a new (potentially novel) compound, these regularities can be applied to the known head noun meaning in order to obtain a meaning for the compound. As discussed in the introduction, this way of obtaining compound meanings is compatible with the literature on conceptual combination [42], [15, 16]. Furthermore, it specifies the claim of the Concept Specialization Model [46], [47] that background knowledge determined which dimensions of the head noun concept are changed to what degree by the modifier: This background knowledge is obtained by learning from experience how the modifier changes the meaning (i.e., the distributional pattern) of head nouns in compounds that are encountered.

This implies that, for each noun in the vocabulary (at least those that were encountered as modifiers before), there have to be at least two representations in the memory: Once as a single word, and once as a function to be applied in compounding, that is as a compounding “instruction”. However, in that our model is not more complex than for example the Concept Specialization Model [46], [47], where it also has to be stored in the background knowledge which slots of which head nouns are filled by the modifier. For this, relying only on a representation of the modifier as a prototype schema is not sufficient; instead, the knowledge about which modifier typically fills which slots of which kind of head noun has to be stored as well. As argued above, Lexical Functions can be interpreted as a formalization of this knowledge.

Obviously, a potential drawback of the Lexical Function model we employed is that the representation of the modifier itself (i.e., its distributional vector) plays no direct role in the compositional process, as it does not affect the computation of the compound vector, and hence the representation of the combined concept. Therefore, the model implicitly assumes that the modifier concept in itself is not employed in conceptual combination. From a psychological perspective, this assumption seems questionable. However, one objection to this point is that it in turn assumes that the modifier lexical functions themselves are completely independent from the modifier concept. Since the lexical functions are estimated from the context in which words appear as modifiers in a compound, this claim is difficult to hold. Especially, as discussed earlier, lexical functions encode the regularities how a modifier typically changes the meaning of head nouns, and this in turn should depend substantially on the modifier concept (however, not as encoded in its distributional vector).

Another argument which can be interpreted in favour of a lexical-function-like approach to the composition of noun compounds has been made in [118], [41]. Booij demonstrates that there are cases where modifiers behave very similar to affixes if they are used within compounds. To put it in the words of [119], “there is so far no semantic theory that allows us to clearly draw the line between compounding and affixation on semantic grounds” (pp. 327–328). One example is the German word Haupt, or the Dutch hoofd (literally head), which in compounds regularly assumes the meaning of main: A Hauptbahnhof is a main station, a Haupteingang is a main entrance, and so on. Lexemes that occur as real words in the lexicon, but adopt a different meaning when used in compounds, are referred to as affixoids or semi-affixes. Such examples can also be found in English, see head office or head nurse. One can, in principle, adopt such an affixoid view of modifiers in compounds in general: According to such a view, the modifier meaning in a compound (or the contribution of the modifier) does not need to be necessarily the same as its meaning when used as a single word. Instead, each word when used as a modifier in principle has the possibility to influence the meaning of a head noun in a very idiosyncratic way. This has to be learned from other contexts in which that word is used as a modifier (compare also [42], [52]). According to such an approach in its generalized form, where modifiers are seen as affixes, the lexical function method is an adequate implementation of modifier contributions to compounds, since it has been shown to capture regular affixation well [7].

This being said, a compositional model that might be psychologically more adequate is the Full Additive model proposed in ([80], see S1 Appendix). In this model, the compound meaning is computed as (10) with c being the n-dimensional compound vector, m and h being the n-dimensional modifier and head vectors, respectively, and A and B being n × n-dimensional weight matrices. As with in the Lexical Function method, the weight matrices are estimated from a training set. As can be seen, in the Full Additive model both constituent vectors play a role in the computation of the compound vector, and the weight matrices specify how much each dimension of each constituents contributes to each dimension of the compound. The difference between the Full Additive and the Lexical Function model is that the Lexical Function model assumes this contribution pattern to be idiosyncratic for each modifier word.

The Full Additive model has properties that possibly make it more psychologically adequate. The most important point is that it allows to compute compound meanings including any modifier and head word for which a distributional vector is available (i.e., a reader has in her mental lexicon), and it thus far more productive than the Lexical Function model that is restricted to compounds whose modifiers were often encountered as modifiers before. Furthermore, the Full Additive model assumes that the dimensional values of both the modifier and head concept directly influence the compound meaning.

However, in our analysis, we still chose to employ the Lexical Function model as a compositional model. The main reason for doing so was a purely empirical one: This model outperformed the Full Additive model in our initial analysis (see S1 Appendix), in that it predicted the human plausibility ratings far better.