Stability in Biology

This situation is similar to evolutionary biology, where stability (and its complement, the rate of evolution) are complex outcomes of multiple factors, including universal and lineage-specific components. Neutral genetic markers evolve at a constant rate dictated by mutation rate [3] resulting in a molecular clock, while nearly neutral markers evolve at a rate determined by mutation and population size [4], reflecting the balance between mutation (production of novelty) and genetic drift (purging variation from the population). However, this is complicated by the non-constancy of mutation rates across the genome, across species and time, being influenced by, among others, the local DNA context, metabolism, life history parameters, age, gender and environmental stress [5], [6]. The various types of natural selection add a supplementary level of complexity. For example, purifying selection will tend to resist change, while positive selection will increase the rates of evolution [7].

Thus, there are highly conserved genes, such as those coding for ribosomal RNA present across the whole of cellular life and covering at least 3.5 billion years [8] or the Pax6 gene (a master gene controlling the cascade leading to eye development) so well conserved that the mouse gene induces eye formation in the fruit fly [9], while the fruit fly homologue genes eyeless and twin of eyeless induce the formation of several eye structures in the frog embryos [10]. At the other end of the spectrum, there are genes which evolve extremely fast, such as some involved in the immune system [11] or male reproductive biology [12], [13], where strong and dynamic natural or sexual selective pressures are acting. Interestingly, there are also stretches of DNA which, despite being very stable in general, have changed a lot in a given lineage, such as the so-called human accelerated regions (HARs; [14], [15]) which have changed dramatically in the lineage leading to us. Several genes involved in microcephaly [16], such as ASPM, Microcephalin and SHH, show faster evolution in primates and especially in the lineage leading to humans, suggesting that they might have been partly responsible for the evolution of increased human brain size. FOXP2, a gene involved in developmental verbal dyspraxia [17] is one of the most conserved genes within vertebrates [18], but modern humans and Neandertals carry a specific variant which differs at 2 positions from the chimpanzee [18], [19]. A proposed explanation for these differences in patterns of stability among genes is represented by the extended complexity hypothesis [20], suggesting that genes that are involved in complex and extensive interactions, and whose products participate in informational processes (transcription, translation and related aspects) and other complex functions, tend to change more slowly.

Stability of Vocabulary

In the case of language, recent work [21]–[23] shows that not all concepts in a list of basic vocabulary – i.e. a standardised list of concepts selected for their universality, the best-known being the Swadesh 100 and 200-words lists [24] – are equally stable. For example, concepts such as “two”, “who”, “tongue”, “night”, “one” and “to die” seem to be extremely stable in the Indo-European language family, showing at most 1 cognate replacement per 10,000 years of language change (depending on the assumed age of the family), while the most unstable meanings, such as “dirty”, “to turn”, “to stab” and “guts”, show up to 9 such replacements during the same period [22]. Moreover, the stability of these concepts has a relatively strong universal component, in the sense that their relative stabilities tend to be conserved across several different language families [22], [23], [25], [26]. An important explanatory factor seems to be the frequency of use of these concepts, with the more frequently used tending to be more stable [22]. Thus, it seems that certain concepts have a set of properties, including their frequency of use, which tend to make them resilient against lexical replacement across language families, time and space, the most stable showing a fidelity comparable to that of genetic systems [23].

Stability of Structural Linguistic Features

The properties and patterns of stability of structural aspects of language, such as the order of subject and verb or the number of consonants in a language, are less well understood. Some authors [27]–[29] suggest that the distributional properties of structural features might inform us about deeper historical relationships than are accessible through the standard comparative method of historical linguistics [30], [31], and they seem, at least in some cases, more resistant to admixture than human genes [32].

In contrast (and in agreement with widespread assumptions in historical linguistics), recent work [26] compared the historical signal and phylogenetic stability of the basic vocabulary to that of structural features in the Indo-European and the Austronesian language families and found that in both families the vocabulary data fitted the comparative-method established family trees much better than the structural data. This suggests that structural features evolve much faster and/or are more influenced by contact phenomena [33] than basic vocabulary. Moreover, the rates of evolution were roughly similar for vocabulary and structural data in both families but the structural features stabilities’ (in contrast to the vocabulary) show very weak correlations across these language families, leading the authors to conclude that they “do not support the existence of a set of universally stable typological features” (p.6). Likewise, other recent work [34] suggests that structural properties are language family-specific, although this work was not directly aimed at studying the stability of structural features but to understanding the regularities governing the temporal dynamics (i.e. correlated evolution) of various aspects of word order. Both these studies suffer from a limited coverage of different language families, potentially undermining the generality of their findings. By considering more language families and structural features, recent work involving the first author [35], [36] seems to reconcile these two views by finding that there is an important universal, cross-language family component to the stability of structural features, but that there is also a non-negligible amount of variation among families. The view that multiple factors, such as universal tendencies, vertical and horizontal processes are requried to explain linguistic diversity has been suggested before (e.g., [27], [37]), but more empirical work using large databases and modern quantitative methods is required for a thorough understanding of this complex interplay [36].

There is currently a vigorous debate concerning the stability of the structural properties of languages discussing whether (i) there are universal, cross-language family tendencies in that some features are more stable than others, or (ii) the stability of a feature is entirely a language family-idiosyncratic property. The first view points to possible universal biases acting on structural features which influence their stability. These biases can be due to communicative pressures, or to extra-linguistic factors (such as neuro-physiologic, cognitive, articulatory and perceptive constraints), or to factors related to the linguistic system itself. As a result, some features might play a more central role in shaping the structural system of a language (alike to the model of the extended complexity hypothesis [20] in biology). The second view instead suggests that “historical accidents” (or “driving factors” in [38]) specific to individual language families are the major determinants of structural change. Of course, there is the third possibility [35], [36], namely that these two views are not mutually exclusive but complementary.

Summary of Paper

The present paper represents an empirical approach to the issues surrounding the stability of structural features. Here, we compare seven different published methods of estimating the stability of structural linguistic features, in order to quantify their overlap and differences. These seven methods propose different definitions of the concept of structural feature stability and different estimation techniques of these stabilities, while using the same large database of language families and features, namely the World Atlas of Language Structures (WALS) [39]. Reviewing the various methods, we found that the seemingly simple concept of structural stability hides an irreducible complexity, mainly due to the prevalence and importance of horizontal processes in language change [40] and the manner in which various proposals acknowledge and quantify them. We believe that conceptual clarity about structural stability is a necessary step in any discussion concerning language change and evolution, and our empirical approach complements theoretical frameworks such as Nichols’ [41].

Most importantly, we found that, despite this variety of conceptualizations and methodological approaches, there is an important agreement between these different stability estimates. This strongly suggests the existence of a universal, cross-language family component, probably due to intrinsic properties of the structural features above and beyond the particular constraints of the specific language families and areas. However, this universal component does not explain the whole range of variation in structural stability, showing that there are also language family-specific factors at work. These findigs might help better understand the interplay between the various “competing forces” and the relationship between different types of stability in particular language families and areas and for particular structural features [38], [41]. We hope that these findings will open the door to a research program aiming at understanding the nature and exact mixture of universal and idiosyncratic components governing structural language change and evolution.

Cysouw, Albu & Dress (2008): the Consistency with Overall Patterns

These authors [43] develop a very original take on the issue of stability, in that their primary interest is in identifying consistent structural features. Such features “are most indicative of the overall structure of a language […] of the typological profile, or ‘genius’ of a language” (p.263) and are identified by comparing their distributional properties with the “averaged” distribution of many features. The fundamental insight is to compute the typological (structural) distances between languages relative to each structural feature and to quantify how accurately the typological distance given by any single feature reflects the overall typological distance given by all features considered simultaneously.

More specifically, the authors start by defining the typological distance between a pair of languages and relative to a feature as being 0 if both languages share the same attested feature value, 2 if they have attested but different values, and 1 if the feature value for at least one language is unattested (missing data). This is extended to a set of features by taking the average of for all features which have attested values in both languages. This leads naturally to a set of distance matrices between all pairs of languages, first, one matrix per feature, , and, second, an overall distance matrix computed for all considered features. The features for which is more similar to are defined as more consistent. The authors propose three ways to quantify this fit between and :

• Mantel’s congruence test, denoted in the following by CM, is based on Mantel’s proposal [44] to compute the similarity between the two matrices and as the proportion of matrices (obtained by randomly permuting ’s rows) which have a higher correlation with . In effect, this method uses the inverse of the p-value derived from the permutations test as the measure of consistency;
• the coherence method (CC) is based on the “excess” of two languages and relative to a third language given a distance matrix , denoted and computed as averaged across all languages for the feature , versus the overall distances . The excess measures the extra distance between languages and when taking a detour through language . With these, the coherence of feature is the ratio of the excess given to the excess given averaged across all possible triplets of languages, ;
• the rank method (CR) is based on the rank of a language relative to another language defined as the number of languages whose distance to is smaller than the distance of to , With these, the rank matrix between languages is defined as , and the coherence of a feature is computed based on the average ranks of languages sharing the same feature values, , where have the same value for as , and is the smallest possible value of the sum in the numerator. Thus, the method quantifies the ranks of languages that share the feature value with averaged across all languages .

Thus, there are three different ways of measuring the consistency of a structural feature with the overall pattern of all features, each quantifying it in a different manner. Given the complexity of these methods, we urge the interested readers to consult the original paper [43] for a better understanding of their details.

The authors report that (i) these quantifications tend to give comparable results across different datasets (with CC being the most resilient), that (ii) their sensitivity to the amount of missing data varies dramatically between methods (with CR being the least sensitive), that (iii) they seem relatively unaffected by the distribution of feature values and, surprisingly, that (iv) they do not inter-correlate very well (except for CC and CR, for which , ).

Finally, and importantly for the present paper, the authors tested the relationship between consistency and genealogical stability by comparing the distances between related and unrelated languages using sets of the most consistent 25%, 50% and 75% features, and found that related languages had significantly lower distances than unrelated languages. This suggests that the most consistent features might be genealogically stable as well, in the sense that they do distinguish between related and unrelated languages, their values tending to be inherited. If such consistent features do indeed reflect the “typological profile” of the languages and if we assume that such profiles tend to be vertically transmitted, then consistent features will also be stable in the genealogical sense, being inherited from ancestor to daughter languages.

Dediu (2011): the Phylogenetic Rates of Evolution

The approach from [35] (denoted in the following as D) estimates the stability of structural features from a Bayesian phylogenetic perspective [45], [46]. More precisely, for a given language family, the observed values of the structural features within this family’s languages together with the tree representing the genealogical relationships between these languages are used to infer the rates at which the structural features have changed in this family. As is specific to Bayesian methods in general [47], [48], this results in a posterior distribution of such rates giving the posterior probability that a particular structural feature changes at a particular rate in this family.

The author uses two methods for estimating the rates of structural change: (i) a method estimating the probability of a transition between two feature values during an infinitesimally short time, implemented in MrBayes 3 [49], and (ii) another method related to maximum parsimony [50] estimating the minimum number of changes required to produce the observed feature values starting from the inferred root value, implemented in BayesLang [35]. It must be highlighted that these phylogenetic methods are agnostic in what concerns the cause for these structural changes: they could include “spontaneous mutation”, borrowing, language shift and various forms of selective pressures acting on language (“driving factors”) such as cognitive, perceptive or articulatory biases. Therefore, non-vertical processes are treated as just another cause of structural change and reflected in the posterior distribution of rates [36].

In order to control for the influence of particular historical classifications, the author used the classifications given by WALS [39] and Ethnologue [42], later [36] extending this to a collection of more accepted classifications [51]. Given that absolute rates of change cannot be directly compared across language families (as it would require the absolute dating of the root proto-languages), they were converted to ranks with features ordered from the most to the least stable. This reduction in measurement level to ordinal ensures comparability not only across family trees but also across different methods. Most importantly, all combinations of the two methods of stability estimation and the three classifications produced highly similar estimates for the rates of structural change [35], [36].

Maslova (2002, 2004): Estimating Transition Probabilities

Elena Maslova [52]–[55] proposed a method to estimate the transition probabilities between the values of a given structural feature using pairs of closely related languages. Basically (for a complete exposition of Maslova’s method see [56] and especially Methods S1 here, where we also give the R code [Script S1] used to implement the method using the WALS data [Dataset S1] ), for a binary structural feature which can take values and , there are four possible transitions in a given time period for a given language: with probability , with probability , does not change with probability , and does not change with probability . With these, the stability of feature is

To estimate and , we need to sample pairs of related languages and compute the divergence rate, , defined as the proportion of such pairs differing for feature . If we denote the frequency of languages with value for feature as , Maslova derives the following equation (see the Supporting Online Information):allowing the estimation of and from at least two such samples.

This method is based on the same fundamental insights [56] as the fully phylogenetic methods discussed above [35], but it uses a much simpler statistical approach and requires stronger assumptions concerning the relationships between the pairs of closely related languages. We have implemented this method in R [57] and estimated the stability of the structural features in WALS [39] using WALS’ genera (the intermediate level between individual languages and language families, such as Germanic and Romance within Indo-European) to provide the sets of closely related languages (these estimates are denoted in the following as M).

Parkvall (2008): Borrowability versus Genealogical Stability

Mikael Parkvall [58] proposes to distinguish between features which have a strong genealogical signal (‘a language needs to be “born with them” in order to have them’) from those that ‘may “come and go as they please’” (p.234). More precisely, he contrasts genealogically stable features defined as “a language either has it or lacks it, but whatever the case, contact or internal development is not going to change much” to unstable features, defined as “an easily borrowable or transferable characteristic, or for that matter, a feature easily gained or lost in contact” (p.235). Thus, it seems that his real focus is not on resistance to change whatever the cause of the change, be it internal (“mutation”-like processes) or external (language contact, selective pressures, random sampling or various types of constraints), but specifically on resistance to borrowing.

This is reinforced by the actual operationalisation of his definition (pp.235–238), which can be summarised as follows. First, he contrasts genealogical units (families and subfamilies from WALS [39]) to areal units (shown in his Map 1 on p.236). Second, for a given feature and a unit he computes the Herfindahl-Hirschman index (or Gini coefficient) defined as.where is the proportion of languages in which have value for the considered feature; in fact, this index is widespread in economics and is closely related to entropy (see Appendix A in [43]). The group’s homogeneity is obtained by taking the reciprocal of the Gini coefficient,

These homogeneities are then averaged over all the considered groups, resulting in the average homogeneity of feature over families (when the groups are genealogical) and areas (when the groups are areal) . The actual measure of stability is the ratio of the two:

When using all the language families available in the WALS, the author obtains a stability estimate which we will denote here as , but he also considered only a subset composed of “only the most widely accepted families” (Algonquian, Austronesian, Bantu, Dravidian, Indo-European, Iroquoian, Mayan, Mongolic, Semitic, Sino-Tibetan, Turkic and Uralic; p. 240) resulting in an estimate denoted here as . Those features scoring high on (and ) are those for which , and thus those for which related languages tend to share the same value as opposed to those for which languages in contact share the same value.

Therefore, we propose that this method should be more appropriately seen as estimating borrowability (and its opposite, resistance to borrowing) and not as stability in the sense of resistance to change. The difference between them is readily seen if we think about a feature which is easy to borrow and yet not very stable.

Wichmann & Holman (2009): Stable Features Tend to “Stay in the Family”

These authors [59] define the stability of a feature as “the probability that a given language remains unchanged with respect to the feature during [a fixed and arbitrary number of years; our note], that is, the feature undergoes neither internal change nor diffusion during the interval” (p.12), being thus explicitly an estimate of the feature’s resistance to change irrespective of the causes of change. They propose three slightly different methods for estimating the relative stabilities of WALS features. However, they conclude (by using computer simulations) that “metric C” performs the best, this is the only one we will describe and use here (denoted in the following as W).

The idea behind “metric C” is “that if one given feature more often tends to have the same value for languages that are related than does another given feature, then the first of the two may be considered to be more stable” (p.16) but the authors also correct for overall tendencies as well. For a feature and a genealogical group consisting of languages for which the feature is attested, they compute the proportion of pairs of languages sharing the same feature value . These proportions are then averaged across groups by weighting each group by resulting in.

Likewise, they compute the proportion of pairs of unrelated languages sharing the same feature value and their stability estimate is obtained by correcting for this.

Thus, given that both and are bounded by and , quantifies how much more similar related languages are than unrelated languages (on average), weighted by the maximum possible such difference. Therefore, is a measure of genealogical stability irrespective of the actual causes of change.

Comparing the Methods

For each of these methods we have extracted the estimated stabilities for each of the 142 structural features in WALS [39], as follows:

• Cysouw, Albu & Dress (2008): for each of their three methods (CM, CC and CR) we have extracted the estimated stabilities from their paper’s Appendix D [43]. We use here the negative of CR to align it with the other methods;
• Dediu (2011): we have extracted the agreed ranks (the scores on the first principal component) for the polymorphic features from Table S7 in the paper’s Electronic Supplementary Material [35]. We use here the negative of this estimate, D, to align it with the other methods;
• Maslova (2002, 2004): we computed the stabilities, M, as described above and as detailed in the Supplementary Material Online;
• Parkvall (2008): to allow comparability with the other methods, we have retained only the estimates for polymorphic features as computed using “all families” (denoted in the following by P) and using only the “most widely accepted families” (P), both extracted from his paper’s Appendix (pp.245–250) [58];
• Wichmann & Holman (2009): we extracted the estimates produced by their “metric C” (denoted in the following by W) from their paper’s Appendix 1 (pp.43–46) [59].

These estimates are reported in Table 1 and each method’s coverage of the 142 WALS features is given in Table 2. P and M cover the most features (136; 97.18%) while D covers only 68 (47.89%): these differences are explained by the minimal requirements of the methods and the threshold of maximally acceptable proportion of missing data used by the different authors. There are 62 (43.66%) shared features covered by all methods, namely (please note that WALS uses unique numeric identifiers for its features, given here in parantheses): Consonant inventories (1), Vowel quality inventories (2), Voicing in plosives and fricatives (4), Uvular consonants (6), Glottalized consonants (7), Lateral consonants (8), Velar nasals (9), Vowel nasalization (10), Front rounded vowels (11), Syllable structure (12), Tone (13), Absence of common consonants (18), Locus of marking in the clause (23), Locus of marking in possessive noun phrases (24), Reduplication (27), Number of genders (30), Definite articles (37), Indefinite articles (38), Distance contrasts in demonstratives (41), Pronominal and adnominal demonstratives (42), Third-person pronouns and demonstratives (43), Gender distinctions in independent personal pronouns (44), Politeness distinctions in pronouns (45), Number of cases (49), Asymmetrical case marking (50), Ordinal numerals (53), Numeral classifiers (55), Position of pronominal possessive affixes (57), Obligatory possessive inflection (58), Possessive classification (59), Nominal and verbal conjunction (64), Perfective/imperfective aspect (65), Past tense (66), Future tense (67), Perfect (68), Morphological imperative (70), Optative (73), Overlap between situational and epistemic modal marking (76), Semantic distinctions of evidentiality (77), Suppletion according to tense and aspect (79), Verbal number and suppletion (80), Order of subject and verb (82), Order of object and verb (83), Order of adposition and noun phrase (85), Order of genitive and noun (86), Order of adjective and noun (87), Order of numeral and noun (89), Order of degree word and adjective (91), Position of polar question particles (92), Position of interrogative phrases in content questions (93), Verbal person marking (102), Order of person markers on the verb (104), Passive constructions (107), Antipassive constructions (108), Applicative constructions (109), Symmetric and asymmetric standard negation (113), Predicative adjectives (118), Nominal and locational predication (119), Zero copula for predicate nominals (120), ‘When’ clauses (126), ‘Hand’ and ‘arm’ (129), M-T pronouns (136). N-M pronouns would also belong to this list if not for P₂.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Estimates of stability given by the various methods, converted to relative ranks (from 0.00 = most unstable, to 1.00 = most stable) for easiness of comparison between methods.

https://doi.org/10.1371/journal.pone.0055009.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Coverage of the WALS features.

https://doi.org/10.1371/journal.pone.0055009.t002

Conceptually, these methods propose quite different approaches to the structural stability of language. Dediu (2011) [35] uses a standard concept from evolutionary biology, in which stability is equated with resistance to change (irrespective of the causes of change) while languages evolve following an assumed tree-like history. Stable features are those with a low rate of change. A related idea, but much simpler and ignoring many possible problematic issues, is proposed by Wichmann & Holman (2009) [59], which take stable features to be those that tend to share values within families rather than across them. Maslova’s method [52]–[55] also shares fundamental insights with Dediu (2011) and Wichmann & Holman (2009) in the sense that stability is understood in a genealogical context. Parkvall (2008) [58] estimates something which would probably be better called non-borrowability rather than stability in the sense that features high on this scale are those shared more within genealogical units than within linguistic areas. Finally, Cysouw, Albu & Dress (2008) [43] describe a method which apparently has no genealogical component, whereby they estimate the consistence of a feature with the overall pattern given by many features.

Pairwise Relationships between Methods

Using all 62 features shared across all methods, the relationships between all pairs of methods are represented in the scatterplots in Figure 1. Table 3 shows the pairwise correlations (Pearson’s and Spearman’s ) between the stability estimates. For each pair of methods, we inspected the scatterplots and regression diganostic plots (using R’s lm() function; Residuals vs Fitted, QQ-plot and Leverage) in order to identify outliers. Given the small number of shared features across all methods, we applied a conservative approach by selecting only those features that were strong outliers for several pairs of methods, identifying the following features: Verbal Number and Suppletion (80), Obligatory Possessive Inflection (58), M-T Pronouns (136), and Front Rounded Vowels (11); see also Figure 1. Without these outliers the correlations do not change much (see Table 4), except for , as it takes an extreme position for almost all outlier features.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Relationship between different stability estimates.

Each panel shows the scatterplot of the stability estimates for the shared features produced by a pair of methods (grey dots) and the identified outliers (red crosses; see text for details). The regression lines with the outliers (red) and without (blue) have been drawn for convenience.

https://doi.org/10.1371/journal.pone.0055009.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Pairwise correlations between stability estimates for all shared features.

https://doi.org/10.1371/journal.pone.0055009.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Pairwise correlations between stability estimates excluding the outliers.

https://doi.org/10.1371/journal.pone.0055009.t004

Figure 2 shows a different view of the relationship between methods: in the 62-dimensional space determined by the shared features, each method represents a single point with coordinates given by the relative ranks of the features as computed by the method. In order to meaningfully compare the feature stabilities across methods, we converted them to relative ranks between 0.0 (most unstable feature) and 1.0 (most stable feature). We did this for each method separately, by first computing the ranks of the method’s actual stability estimates for all the features that the method provided estimates for, and then by normalizing these ranks between 0 and 1 using the formula where is a given feature’s rank and and are the smallest and largest ranks respectively. In this space we computed all pairwise Euclidean distances between the methods and projected these distances on two dimensions using classical multidimensional scaling (MDS) [60], resulting in Figure 2. A small distance between two methods means that they tend to estimate the same relative stabilities for all features, while a large distance signals disagreements between methods. The maximum possible distance in this space is but the distances between methods are between 1.34 and 3.43 with a mean of 2.43, suggesting again that the methods agree better than expected by chance. This was confirmed by randomly generating 10,000 sets of seven points in this 62-dimensional space and comparing the distribution of these generated distances to the observed distances between methods: both min and mean observed distances are much smaller than expected (), while the maximum distance is also smaller but within the distribution of maximum random distances (). It can be seen that CC and CR form a tight cluster, as do P₂ and W. Further, D and M are relatively close together, while CM is a clear outlier.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Relationships between methods.

The distances between methods computed in the 62-dimensional space defined by the relative ranks of all shared features projected using classic Multidimensional Scaling (MDS). The results excluding the outlier features are extremely similar.

https://doi.org/10.1371/journal.pone.0055009.g002

Principal Components Analysis

To better understand the relationships between the stability estimates produced by these methods we have conducted a Principal Components Analysis [61] both on the full set of shared features and on the set excluding the outliers, using the actual stability estimates provided by each method.

On the full set of 62 shared features, the first four Principal Components explain in total 89.2% of the variation (Table 5). explains 48.3% and represents the agreement between all methods (their loadings have the same sign). explains 17.2% and contrasts CC, CR and D on one hand, and CM, P₁, P₂ and W on the other (excluding M, which has a loading close to zero on this component). explains 13.5% and makes a further distinction between the three methods in [43] (CM, CC and CR) and the two methods in [58] (P₁ and P₂). Finally, explains 10.2% and contrasts D, W and M with the other methods.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Principal components analysis on the full set of 62 shared features.

https://doi.org/10.1371/journal.pone.0055009.t005

The first principal component, explaining by far most of the variance (48.3%), represents the agreement between all these highly different methods. The following components also make interesting distinctions, such as the grouping together of the strongly phylogenetic method D with two of the strongly non-genealogical methods CC and CR (component 2), the identification of [58]’s special concept of “borrowability” (P₁n and P₂) versus [43]’s “consistency” (CM, CC and CR) (component 3), and the recognition that D, W and M methods are fundamentally similar, even if differing widely in details (component 4).

When excluding the outliers, the first four principal components explain 90.5% of the variation (Table 6). The first principal component, , explains 55.8% of the variation and represents the agreement between all methods with similar loadings to the previous case. explains 16.1% and distinguishes CC, CR, D and M on one hand, from the CM, P₁, P₁ and W, on the other, in a pattern similar to the previous case. However, (12.2% of the variance) and (6.8%) differ from the ones found using all shared features; still distinguishes [43]’s “consistency” (but not [58]’s “borrowability”).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Principal components analysis excluding the outliers.

https://doi.org/10.1371/journal.pone.0055009.t006

Figure 3 (left panel) shows all shared features in the space, capturing together 65.5% of the variation between methods. Features that cluster together are features that show similar stability estimates across methods. The right panel compares the stability of the various WALS areas and shows that Word Order features tend to be the most stable, with Phonology covering the whole spectrum. An one-way ANOVA shows that the areas differ in their average stability (, ), but a post-hoc pairwise comparison using Tukey’s HSD shows that only Word Order – Nominal Categories survive the multiple testing correction (adjusted ), most probably due to the small numbers of features included.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Distribution of features’ stabilities across methods.

Left panel: a ll shared features (given by their numeric WALS unique ID; see Table 1) plotted on the . Right panel: the distribution of the stability across the WALS areas, with the number of features of each type shown on the right. represents the strong inter-method agreement and varies from the unstable (left) to stable (right); the actual scales of the axes are arbitrary. The colours and symbols represent the WALS areas (see Table 1 for details), with the open diamond representing all shared features together. The results excluding the outlier methods are extremely similar.

https://doi.org/10.1371/journal.pone.0055009.g003

The Agreement and Differences between Methods

The pairwise correlations and the Principal Components Analysis strongly suggest that there is an important agreement between these different methods in what concerns the stability of the shared features.

Table 7 shows the shared features sorted from the most stable to the most unstable by their scores on the first principal component when using all shared features, , and then by the first principal component when excluding the outlier features, . The correlation between and is , .

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. The shared features sorted by their agreed stabilities from the most stable to the most unstable.

https://doi.org/10.1371/journal.pone.0055009.t007

Figure 4 shows the shared features ordered by their median relative rank across all methods, an indication of the agreement between methods for each feature (the interquartile range, IQR), as well as the actual estimate given by each individual method. Table 8 shows the features ordered by the disagreement between methods (IQR). It can be seen that despite a clear overall concordance between methods (as shown by the first principal component), the agreement is far from perfect, and there are clear differences between methods both overall (reflected in the second, third and fourth Principal Components, and in the patterns of inter-method correlations and distances), and in what concerns the estimated stability of individual features.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. The features’ stabilities.

The stabilities (as relative ranks from 0.0 = most unstable to 1.0 = most stable) of the shared features as estimated by all methods. Shown are the median stability (black thick lines), the interquartile range (IQR; light gray) and the individual method estimates (D, 1, 2, W, A, C, R, and M – see legend for details). The features with a significantly smaller or larger IQR than expected by chance are marked with red “<” and blue “>” symbols respectively on the right-hand side of the figure, with the number of symbols being one for , two for and three for ; please note that this is before the multiple testing correction, after which only the features with , 11 and 87, survive (see text for details). The features are represented by transparently abbreviated names derived from their full WALS names (see Table 8) and their WALS unique IDs.

https://doi.org/10.1371/journal.pone.0055009.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Differences in stability among methods for individual features.

https://doi.org/10.1371/journal.pone.0055009.t008

The IQR varies between 0.08 and 0.89 with a mean of 0.30, and while some features show very little disagreement between methods (such as 87: Order of Adjective and Noun, 86: Order of Genitive and Noun and 18: Absence of Common Consonants), for some (such as 11: Front Rounded Vowels, 129: ‘Hand’ and ‘Arm’ and 136: M-T Pronouns) the methods disagree almost completely. In order to better understand these IQR values, we computed their expected distribution by randomizing 10,000 times the stability estimates across features and computing the IQR between the methods; the distribution is relatively normal (as judged from the QQ-plot) with a mean of 0.45 and standard deviation of 0.14. For each feature we them compared its IQR to this expected distribution and we found that, after Holm’s correction for multiple testing [62], only feature 87: Order of Adjective and Noun has an IQR less than expected (adjusted ), and feature 11: Front Rounded Vowels has an IQR much larger than expected (adjusted ). Without multiple testing correction, 24 features have a smaller IQR than expected at an -level of 0.05, but only 11: Front Rounded Vowels has a larger IQR (see Figure 4 for details). For future research it will be interesting to clarify for individual features why the methods disagree and what these disagreements mean from a theoretical perspective.