A Complex Network Approach to Stylometry

Diego Raphael Amancio

doi:10.1371/journal.pone.0136076

Abstract

Statistical methods have been widely employed to study the fundamental properties of language. In recent years, methods from complex and dynamical systems proved useful to create several language models. Despite the large amount of studies devoted to represent texts with physical models, only a limited number of studies have shown how the properties of the underlying physical systems can be employed to improve the performance of natural language processing tasks. In this paper, I address this problem by devising complex networks methods that are able to improve the performance of current statistical methods. Using a fuzzy classification strategy, I show that the topological properties extracted from texts complement the traditional textual description. In several cases, the performance obtained with hybrid approaches outperformed the results obtained when only traditional or networked methods were used. Because the proposed model is generic, the framework devised here could be straightforwardly used to study similar textual applications where the topology plays a pivotal role in the description of the interacting agents.

Citation: Amancio DR (2015) A Complex Network Approach to Stylometry. PLoS ONE 10(8): e0136076. https://doi.org/10.1371/journal.pone.0136076

Editor: Peter Csermely, Semmelweis University, HUNGARY

Received: May 27, 2015; Accepted: July 22, 2015; Published: August 27, 2015

Copyright: © 2015 Diego Raphael Amancio. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: The author acknowledges support from São Paulo Research Foundation (FAPESP) (grant number 14/20830-0). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The author has declared that no competing interests exist.

Introduction

The human language represents a major factor responsible for the success of our species; and its written form is one of the main expression used to convey and share information. Owing to the ubiquity of language in several contexts, many linguistic aspects have been studied via the application of methods and tools borrowed from diverse scientific fields. As a consequence, several findings related to the origins, organization and structure of the language have been unveiled. One of the most fundamental patterns arising from statistical analysis of huge amounts of text is the Zipf’s law, which states that the frequency of the words decreases inversely to their rank [1–3]. Other fundamental recurrent pattern is the Heap’s law, which states that the vocabulary size grows slowly with the number of tokens of the document [4–6]. More recently, concepts from Physics have been applied to model several language features. For example, with regard to long-scale properties, concepts from dynamical systems have successfully been employed to compare the burstiness of the spatial distribution of words in documents [7, 8]. In a similar fashion, the spatial distribution of words in documents and analogous systems have also been studied in terms of level statistics [9], entropy [10] and intermittency measurements [11]

A well-known approach to study written texts is the word adjacency network model [12–14], which considers short-scale textual properties to form the networks. In this representation, relevant words conveying meaning are modelled as nodes and adjacency relationships are used to establish links. Using this model, several characteristics of texts and languages have been inferred from statistical analyses performed in the structure of networks [15]. Language networks have been increasingly employed to understand theoretical linguistic aspects, such as the origins of fundamental properties [16] and the underlying mechanisms behind language acquisition in early years [17]. In practical terms, networks have been applied in the context of machine translations [18], automatic summarization [19], sense disambiguation [20, 21], complexity/quality analysis [22, 23] and document classification [24]. Despite the relative success of applying networks concepts to better understand language phenomena, in many real-world applications attributes extracted from networked models have not contributed to the advancement of the state of art. For example, when one considers the document classification task, a strong dependency of network features on textual characteristics have been observed. However, the best performance are still achieved with traditional statistical natural language processing features. In this context, the present study address this problem by devising methods that effectively take advantage of network properties to boost the performance of the textual classification task. Here I focus on the text classification based on stylistic features, where a text is classified according to stylistic marks left by specific authors [25] or literary genres [26]. Upon introducing a hybrid classifier relying on the fuzzy definition of supervised pattern recognition methods [27], I show that the performance of style-based classifications can be significantly improved when topological information is included in traditional models. Given the generality of the proposed method, the framework devised here could be applied to improve the characterization and description of many related textual applications.

Representing texts as networks

There are several ways to map texts into networks [12, 15, 19, 28–30]. The most suitable form depends on the context of the application. In occasions where the semantics is relevant, the words sharing some semantical relationship are linked [20, 22, 31–33]. In a similar fashion, other semantic-based models link the words appearing in a given context [34, 35] (e.g. in the same sentence or paragraph). A general model for establishing significant “semantical” links between co-occurring elements was devised in [36]. More specifically, the model considers the existence of a set of elements V = {v₁, …, v_n}, which may occur in one or more sets of a given collection ξ = {S₁, …, S_N}. Given two elements α ∈ V and β ∈ V, the model computes the probability p_r that more than r sets in ξ contain both elements α and β as (1) where p_r(j) is the probability that α and β co-occur exactly j times in the same set. Given a confidence level p₀, the strength of the link between α and β is s = log(p₀/p_r).

In applications where the style (or structure) plays an important role, the links among words are established according to syntactical relationships [19, 30]. A well-known approach for grasping stylistic features of texts is the word adjacency model [13, 37, 38], which basically connect adjacent words in the text. Differently from the model devised in [36], the word adjacency model captures the stylistic features of texts [39]. It has been shown that the adjacency model is able to capture most of the syntactical links with the benefit of being language independent. Despite being a simplification of the syntactical analysis, the adjacency model has been employed in several contexts because the topological properties of word adjacency and syntactical networks are similar. Such high degree of similarity can be explained by the fact that most of the syntactical links occurs among neighbouring words [30]. In the current paper, the traditional word adjacency representation was adopted.

Before mapping the text into a word adjacency network, some pre-processsing steps are usually applied. First, words conveying low semantic content, such as articles and prepositions, are removed from the text. These words, referred to as stopwords, are disregarded because they just serve to link content words. As a consequence, they can be straightforwardly replaced by network edges. The remaining words are then lemmatized, i.e. they are transformed to their canonical forms. To assist the lemmatization process, all words are labeled with their part-of-speech tags. Particularly, in this study, I used a model based on maximum entropy [40]. To exemplify the construction of a word adjacency network, I show in Table 1 the pre-processing steps performed in a short text. The corresponding network is shown in Fig 1.

Download:

Table 1. Pre-processing steps applied to construct networks from texts.

https://doi.org/10.1371/journal.pone.0136076.t001

Download:

Fig 1. Word adjacency model.

Example of word adjacency network created from the poem “In the Middle of the Road”, by Carlos Drummond de Andrade. The pre-processing steps performed to generate this word adjacency network are shown in Table 1.

https://doi.org/10.1371/journal.pone.0136076.g001

Topological characterization of networks

A network can be defined as where denotes the set of nodes and denotes the set of edges, which serve to link nodes. An unweighted network can be represented by an adjacency matrix A = {a_ij}, where each element a_ij stores the information concerning the connectivity of nodes i and j. If i and j are connected, then a_ij = 1. Otherwise, a_ij = 0. Note that, in a undirected network devoid of self loops, A^T = A. Currently, there are several network measurements available to characterize the topology of complex networks [41]. Here, I describe the main measurements applied for text analysis:

Degrees: the degree (k) is the number of edges connected to the node, i.e. k_i = ∑_j a_ij. A relevant feature related to the degree that has been useful for text analysis are the are the average node degree and the standard deviation of the neighbours, which are given by (2) (3) In text networks, both k⁽ⁿ⁾ and Δk⁽ⁿ⁾ have been useful to quantify the structural organization of texts [42].
Accessibility: the accessibility measurement is a extension of the node degree centrality [43]. It is defined as a normalization of the diversity measurement [43], which quantifies the irregularity of a accessing neighbours through self-avoiding random walks [43]. To define the accessibility, let P_h(i, j) be the probability of a random walker starting at node i to reach node j in exactly h steps. The heterogeneity of access to neighbours can be quantified with the diversity measurement: (4) Given Eq 4, the accessibility (α) is computed as (5) It can be shown that the accessibility is bounded according to the relation 0 ≤ α^(h) ≤ n_h, where n_h is the number of nodes at the h-th concentric level [44]. An example of the computation of the accessibility in a small network is shown in Fig 2. In the example, nodes 2, 3, 4 and 5 belong to the first concentric level and nodes 6, 7, 8, 9 and 10 belong to the second concentric level. When one considers a regular access to the second level (red configuration), the accessibility corresponds to the total number of nodes located at the h-th concentric level. When some nodes are more accessed than others, the accessibility decreases because less nodes are effectively accessed.
In textual networks, it has been shown that the accessibility is more advantageous than other traditional centrality measurements as it is able to capture more information at further hierarchical levels [44]. This measurement has been successfully applied to detect core concepts in texts [19]. Furthermore, it has also been employed to generate informative summaries [19]. A dependency of the distribution of this measurement with stylistic features of text was observed in [45].
Betweenness: the betweenness is a centrality measurement that considers that a node is relevant if it is highly accessed via shortest paths. If n_sit is the number of geodesic paths between s and t passing through node i; and n_st is the total number of shortest paths linking s and t, then the betweenness is defined as: (6) In word adjacency networks, high frequent words usually take high values of betweenness. However, some words may act as articulation points whenever they link two semantical contexts or communities [46]. It has been shown that the betweenness is able to identify the generality of contexts in which a word appears [47]. More specifically, domain-specific words tend to assume lower values of betweenness when compared with more generic words.
Assortativity: several real-world networks are formed of nodes with a specific type of classification. For example, in social networks, individuals may be classified by considering their age, sex or race. When analyzing the connectivity patterns of networks, it might be relevant to study how distinct classes connect to each other. This type of analysis is usually performed with the so-called assortativity measurement [48]. The assortativity can be computed as the Pearson correlation coefficient (r): (7) where e is the total number of edges. In word adjacency networks, the assortativity quantifies how words with distinct frequency appear as neighbours.
Clustering coefficient: the clustering coefficient (C) quantifies the local density of neighbours of a given node. The local definition of the clustering coefficient is given by the fraction of the number of triangles among all possible connected sets of three nodes: (8) Similarly to the betweenness, the clustering coefficient is useful to detect words appearing in generic contexts [47]. However, differently from the betweenness, the clustering coefficient analyzes only the local neighbourhood of nodes.
Average shortest path length: the average shortest path length (l) is the typical distance between any two nodes in the network. This measurement was used in this paper because it has been useful in stylistic-based applications [47]. In texts, the average shortest path length quantifies words relevance. More specifically, according to this measurement, the most relevant words are those that are close to the hubs.

Download:

Fig 2. Accessibility in a toy network.

Probabilities of transition from node 1 considering h = 2 steps for two distinct configuration of links. The first configuration considers only the red edges and the second one considers both red and blue edges. Note that, in the first configuration, the probability to reach any node at the second level is the same. In this case, . When blue edges are included, nodes 7 and 9 tend to receive more visits than the other nodes, according to the considered probabilities. For this reason, the effective number of accessed nodes drops to .

https://doi.org/10.1371/journal.pone.0136076.g002

Most of the measurements described in this section are local measurements, i.e. each node i possesses a value , where . For the purposes of this paper, it is necessary to sum up the local measurements. The most natural choice is to characterize the documents by computing the average , where stands for the average computed over the N distinct words of the text. A disadvantage associated with this type of summing procedure is that all words receive the same weight, regardless of their number of occurrences in the text. To avoid this potential problem, I also computed the average value obtained when only the η = 50 most frequent words are considered. The standard deviations and the skewness were also used to characterize the documents.

A drawback associated to the computation of local topological measurements in word adjacency networks is the high correlation found between these measurements and the node degree (i.e. the word frequency). To minimize this correlation, the following procedure was adopted. Each of the measurements was normalized by the average obtained over 30 texts produced using a word shuffling technique, where the frequencies of words are preserved. If and are the average and deviation obtained over the random realizations, the normalized measurement X and the error ϵ(X) related to X are (9) (10)

Traditional stylistic features

Frequency of words and characters

Traditional methods usually perform statistical analysis using specific textual features [49]. An important contribution to the stylometry was introduced by Mosteller and Wallace [50], which showed that the frequency of function words (such as any, of, a and on) is useful to characterize the style in texts. Frequent words have also been used in strategies devised by physicists where the distance of frequency ranks was used to compute the similarity between texts [51–53]. Another relevant feature for characterizing styles in texts is the frequency of character bigrams (i.e. a sequence of two adjacent characters). These attributes have proven useful e.g. to detect the stylistic marks of specific authors [54].

Intermittence

The uneven spatial distribution of words along texts is a feature useful for characterizing the style of texts [10]. The quantification of the homogeneity of the distribution of words along texts can be performed by using recurrence times, a standard measure employed to study time series [55]. In texts, the concept of time is represented in terms of the number of words occurring in a given interval. For each word i, the recurrence time T_j is defined as the number of words appearing between two successive occurrences of i plus one. For example, the recurrence times of the word “stone” in the lemmatized text shown in Table 1 are T₁ = 1, T₂ = 3, T₃ = 3, T₄ = 11, T₅ = 1 and T₆ = 5. If a word i occurs N_i times in a text comprising N_T words, it generates a sequence of N_i − 1 recurrence times {T₁, T₂, …, T_{N_i−1}}. In order to consider the time T_f until the first occurrence of i and the time T_l, the number of words between the last occurrence of i and the last word of the text, the recurrence time T_N = T_f + T_l is added to the set of recurrence times of word i. As such, 〈T〉 = N_T/N_i, where 〈⋅〉 is the average over distinct T_j’s. Note that the average recurrence time 〈T〉 does not provide additional information, since it only depends on the frequency. The intermittence (or burstiness) of the word i is obtained from the coefficient of the variation of the recurrence times: (11) It has been shown that the intermittence has been useful to identify core concepts in texts, even when a large corpus is not available. Relevant words usually take high values of intermittency, i.e. I ≫ 1. Stopwords, on the other hand, are evenly distributed along the text [45]. To illustrate the properties of the intermittency measurement, Fig 3 shows the spatial distribution of the words long and Hobson in the book “Adventures of Sally”, by P.G. Wodehouse. The frequency of these words are similar, yet their values of intermittency are quite different. Note that, while the distribution of long is relatively homogeneous, Hobson is unevenly distributed along the text. Because bursty concepts are the most relevant words [10], in the example of Fig 3, Hobson would be considered a relevant character in the plot.

Download:

Fig 3. Intermittence of words.

Profile of the spatial distribution of long (N_i = 44 and I_i = 1.02) and Hobson (N_i = 45 and I_i = 3.40) in the book “Adventures of Sally”, by P. G. Wodehouse. Because Hobson is unevenly distributed along the book, this word take high values of intermittency.

https://doi.org/10.1371/journal.pone.0136076.g003

Supervised classification

In this section, I describe the two methods employed to combine distinct strategies of classification. The objective here is to combine evidences from both traditional and CN-based methods. Before introducing the methods for combining classifiers, the main concepts concerning the supervised learning task are presented.

In a typical supervised learning task, two datasets are employed: the training and the test dataset. The training dataset X_tr = {(x₁, y₁), (x₂, y₂), …, (x_l, y_l)} is the set of instances whose classes are known beforehand. The first component of the tuples x_i = (f₁ = a₁, f₂ = a₂, …) represents the values of attributes used to describe the i-th instance. The second element y_i ∈ = {y₁, y₂, …} represents the class label of the i-th training instance. In the supervised learning task, the objective is to obtain the map x ↦ y. The quality of the map obtained is evaluated with the test dataset X_ts = {(x_l, y_l), (x_l+1, y_l+1) …, (x_l+u, y_l+u)}. The technique used to evaluate the performance of the classifiers used in this study is the well-known 10-fold cross-validation [56].

An usual procedure in classification tasks is the quantification of the relevance of each attribute for the task. To quantify the relevance of attributes to discriminate the data, several indices have been proposed [56]. A well-known index, the information gain, quantifies the homogeneity of the set of instances in X_tr when the value of the attribute is specified [57]. Whenever a single class prevails when the value of an attribute f_k is specified, the information gain associated to f_k takes high values. Mathematically, the information gain Ω is defined as (12) where ℋ(X_tr) is the entropy of the training dataset X_tr and ℋ(X_tr|f_k) is the entropy of the training dataset when the k-th attribute is specified. The quantity ℋ(X_tr|f_k) is computed as (13) where V(f_k) is the set comprising all values taken by f_k in X_tr.

Hybrid classifier

To define this classifier, consider the following definitions. Let m_ij be the strength associating the i-th instance to the j-th class, where 0 ≤ m_ij ≤ 1. The term m_ij, henceforth referred to as membership strength, can be interpreted as the likelihood of instance i to belong to class j. Note that the quantification of m_ij depends on the classifier being used. A detailed description of methods for computing the membership strength can be found in [27, 58–61]. Let be the membership strength obtained when topological features of complex networks are used and the membership strength obtained when traditional statistical features are analyzed. According to the hybrid strategy, the combination of both topological and traditional methods is achieved according to the following convex combination (14) where λ ∈ [0, 1] accounts for the weight associated to the topological strategy. Note that the combination of evidences performed in Eq 14 yields a membership strength ranging in the interval [0, 1]. The final decision is then made according to the rule (15) where denotes the correct class label associated to the i-th test instance. In practical applications, a screening on the λ can be useful to find its best value. However, this process might be computationally unaffordable for very large datasets. In this case, the process of finding the adequate value of the parameter can be made via application of optimization heuristics [62, 63].

To illustrate how the decision is performed with the hybrid algorithm, Fig 4 shows the classification of a text modelled as a network. In the top panel, the central network is the text whose class is to be inferred. The left and right networks represent two networks in the training dataset. The node color denotes the node label, i.e. the word associated to the node. In this example, two texts are semantically similar if they share the same words. In the central panel, the scenario where the hybrid classified is set with λ = 0.15 is considered. Because λ is close to zero, the classification is mostly based on the number of shared words. In fact, the decision boundary in this case is created so that the test instance is classified in the same class as the right training instance. The bottom panel illustrates the decision made with λ = 0.85. Now, the topological features of the text is the most prevalent feature used for the classification. As a consequence, the boundary decision moves to the right side so that the test instance is classified as the same class as the left training instance. Note that, the test instance as belonging to class c₁ because the r₁ and r_? are topologically similar.

Download:

Fig 4. Hybrid classifier.

Example of classification based on the hybrid classifier. In the top panel, the network r_? might assume two possible classes: c₁ and c₂. An example for each of these classes is provided (see networks r₁ and r₂). In the central panel, the decision boundary obtained for λ = 0.15 is shown. Because λ takes a low value in this case, the decision is mainly based on the number of shared nodes (words). As a consequence, r_? is classified as belonging to class c₂. For higher values of λ, the topological features of texts takes over. In the bottom panel, r_? is classified as belonging to class c₁ because r_? and r₁ are topologically similar.

https://doi.org/10.1371/journal.pone.0136076.g004

Tiebreaker classifier

Similarly to the hybrid classifier, this classification scheme uses as attributes both traditional and topological features of texts modelled as networks. The objective of this approach is to use the topological attributes only when the classification performed with traditional features is not reliable, as revealed by the values of membership strength. Consider that the two most likely classes to which the unknown instance belongs are j and k, according to traditional features. As a consequence, , for each class l ∈ − {i, j}. Let Δ be the difference between the probabilities associated to the two most likely classes, i.e. . If such difference surpasses a given threshold θ, the tiebreaker performs the classification by using only traditional attributes. Conversely, topological attributes are employed to infer the class of the unknown instance. Equivalently, (16) Note that the threshold θ ultimately decides which attributes (traditional or topological) are used to perform the classification. An example of classification using this approach is shown in Fig 5.

Download:

Fig 5. Tiebreaker classifier.

Example of classification based on the tiebreaker classifier. In the left panel, the gray instances are the test instances that should be classified with the class labels red circle or blue asterisk. In this case, traditional attributes were used. Note that the square is significantly far from the decision boundary (dashed line). Therefore, the classification of this instance does not demand the use of topological features because Δ ≥ θ in Eq 16. Differently, the pentagon is located on the decision boundary. Because Δ < θ in this case, topological attributes are used to perform the classification. According to the topological attributes (see right panel), the test instance represent by the pentagon is classified as belonging to the blue class.

https://doi.org/10.1371/journal.pone.0136076.g005

Results and Discussion

The effectiveness of the combination of traditional textual features and topological measurements of networks was evaluated in the context of two natural language processing tasks. Both tasks rely upon the characterization of stylistic marks of texts. The studied tasks were the authorship attribution and genre detection problems. The classifiers chosen to compound the combining techniques were: kNN, Support Vector Machines (SVM), Random Forest (RFO) and Multilayer Perceptron (MLP). These classifiers were chosen because they usually yield good accuracy rates with default parameters [64].

Authorship recognition

In the authorship attribution task, the objective is to identify the authors of texts whose identity is lacking [65]. The dataset employed here comprises books written by eight authors, as shown in Table 2. To show how the topological properties of the networks modelling books can be useful to improve the characterization of authors’ styles in texts, the following combination of attributes were considered:

INT+CN: the intermittence of stopwords were considered along with the topological measurements of the network modeling the text. As stopwords, I considered all the words that appeared at least once in all books of the dataset. According to the dataset, there is a total of 340 stopwords.
FR+CN: the frequency of stopwords were considered along with the topological attributes of the networks.
BG+CN: the frequency of character bigrams were considered along with the topological attributes of the networks. All the possibilities of character bigrams (634) were considered in this analysis.

Download:

Table 2. List of books employed in the authorship recognition task.

https://doi.org/10.1371/journal.pone.0136076.t002

For each combination of attributes, the gain in performance when the topology is considered as an additional feature is represented as: (17) where Γ_H(x) denotes the accuracy rate obtained with either the hybrid or tiebreaker classifier and x is the corresponding parameter employed, i.e. λ for the hybrid classifier and θ for the tiebreaker classifier. Fig 6 shows the values of ΔΓ(λ) when the kNN is used to compose the hybrid classifier. The results obtained for additional values of the parameter k are shown in Table 3. Fig 6 also shows the maximum accuracy rate obtained with the variation of λ: (18)

Download:

Fig 6. Accuracy rate for the hybrid classifier.

Relative accuracy rate as a function of the topological weight (λ) obtained with the k nearest neighbors. The attributes employed were: (a)-(c): intermittence; (d)-(f): stopwords; (g)-(i): characters. The parameters k of the k-nearest neighbors were: k = 3 in (a), (d) and (g); k = 4 in (b), (e) and (h); and k = 5 in (g), (f) and (i). As it turns, there is an improvement of the accuracy rates when traditional methods are combined with the technique based on the topological analysis of complex networks.

https://doi.org/10.1371/journal.pone.0136076.g006

Download:

Table 3. Accuracy rates obtained with hybrid classifier based on the kNN technique.

https://doi.org/10.1371/journal.pone.0136076.t003

With regard to the INT+CN combination, the addition of topological features of texts yielded accuracy rates much higher than the ones obtained with intermittency features alone, since ΔΓ_max > 2 for k = {3, 4, 5}. Note that, in this case, the network approach alone performed better than the method based solely on intermittency features, because ΔΓ(λ = 1) > 1. This observation might explain the high gain obtained when networks are included as an additional feature. As for the FR+CN combination, the highest gain in performance was ΔΓ_max = 1.197, which was obtained for k = 5. The lowest gain in accuracy occurred for the BIG+CN combination. In this case, the maximum gain was ΔΓ_max = 1.070 for k = 5. When other classifiers were used to compose the hybrid classifier, similar results have been found (see Table 4): the highest and lowest improvement in performance occurred for the INT+CN and BIG+CN combinations, respectively. In general, the hybrid classification that combined traditional statistical features and topological measurements of networks improved the classification performance in the authorship recognition task. Interestingly, in several cases, the combination outperformed the accuracy rates obtained when the compounding strategies were analyzes separately, i.e. Γ_H(λ ∈]0, 1[) > Γ_H(λ = 0) and Γ_H(λ ∈]0, 1[) > Γ_H(λ = 1).

Download:

Table 4. Accuracy rates obtained with the hybrid classifier based on the SVM, RFO and MLP methods.

https://doi.org/10.1371/journal.pone.0136076.t004

Traditional and topological attributes of texts were also combined using the tiebreaker classifier. The results obtained with the kNN classifier for k = {3, 4, 5} are shown in Fig 7. The results obtained for other values of the parameter k are displayed in Table 5. The tiebreaker classifier built using the INT+CN combination provided an increase in performance of up to 31%. The maximum gain, obtained with the combinations FR+CN and BG+CN, were respectively 34.2% and 4.8%. With regard to the other classifiers compounding the tiebreaker technique, the results obtained are shown in Table 6. Interestingly, there was no improvement when the SVM was used (ΔΓ_max = 1). Differently, when the RFO classifier was used, an improvement of performance was observed in all three combinations of attributes. Finally, the tiebreaker classifier built with the MLP only provided a gain in accuracy for the combination INT+CN. While providing a gain in performance in several scenarios, the tiebreaker technique usually performed worst than the hybrid classifications performed with the same compound classifiers and parameters.

Download:

Fig 7. Accuracy rate for the tiebreaker classifier.

Relative accuracy rate as a function of the threshold (θ) obtained with the tiebreaker algorithm applied to the the k nearest neighbors. The attributes employed were: (a)-(c): intermittence; (d)-(f): stopwords; (g)-(i): characters. The parameters k of the k-nearest neighbors were: k = 3 in (a), (d) and (g); k = 4 in (b), (e) and (h); and k = 5 in (g), (f) and (i). As it turns, there is an improvement of the accuracy rates when traditional methods are combined with the technique based on the topological analysis of complex networks.

https://doi.org/10.1371/journal.pone.0136076.g007

Download:

Table 5. Accuracy rates obtained with tie-breaker classifier based on the kNN technique.

https://doi.org/10.1371/journal.pone.0136076.t005

Download:

Table 6. Accuracy rates obtained with the tie-breaker classifier based on the SVM, RFO and MLP methods.

https://doi.org/10.1371/journal.pone.0136076.t006

The results in Tables 3–6 confirms that the topology of networks modeling texts is able to improve the characterization of texts because such textual description grasps relevant patterns that are mostly disregarded by traditional statistical approaches. This can be confirmed by optimized results obtained for λ* > 0 and θ* > 0. To better understand the factors behind the discriminability power of the network model, the relevance of each topological for discriminating authors was computed with the information gain criterion. The best topological features were the standard deviation of the accessibility at the third level (Ω(Δα^{(h = 3)}) = 1.07), the standard deviation of the average neighboorhood degree (Ω(Δk⁽ⁿ⁾) = 1.05) and the average clustering coefficient (Ω(〈C〉) = 0.60). An example of discriminability provided by the topological strategy is shown in Fig 8.

Download:

Fig 8. Authorship recognition.

Discriminability of authors obtained with two topological features of complex networks modelling texts. Note that, using only two features, it was possible to separate e.g. Alger from Melville. According to the information gain criterion, the most relevant network features for the authorship identification task were the standard deviation of the accessibility computed at the third level (Ω(Δα^{(h = 3)}) = 1.07) and the standard deviation of the average neighboorhood degree (Ω(Δk⁽ⁿ⁾) = 1.05).

https://doi.org/10.1371/journal.pone.0136076.g008

Style identification

In this section, I investigate if the topological features of complex networks are useful to complement the traditional textual characterization for the style identification task [66]. The dataset used was the Brown corpus [67], which comprises documents that are classified either as informative prose (e.g. press reportage, popular folklore and scientific manuscripts) or imaginative prose (e.g general fiction, romance and love stories). Differently from the authorship attribution task, the objective here is to cluster together documents written using the same style, regardless of their authorship.

The results obtained for the style identification in the Brown corpus using the hybrid technique are shown in Tables 7 and 8. Considering the INT+CN combination, the highest gain in performance were 29.0% and 25.8% for the kNN (k = 4) and SVM methods. Minor improvements in accuracy were observed for other combinations, which might be explained by the high discriminability rates observed for the traditional techniques based on the frequency of stopwords (Γ = 95.6%) and character bigrams (Γ = 94.05%). The results obtained with the tiebreaker technique were inferior to those obtained with the hybrid technique (results not shown). All in all, the results confirm that the inclusion of topological attributes might also improve the characterization of documents in the context of stylistic-based classification tasks. In this task, the most relevant topological features were the skewness and deviation of the clustering coefficient (Ω(γ(C)) = 0.172 and Ω(ΔC) = 0.157). A visualization of the discriminability provided by topological features is shown in Fig 9. A linear discriminant analysis [68] was employed to generate the figure.

Download:

Table 7. Best relative accuracy rate Γ_max obtained with the hybrid classifier based on the k-nearest neighbors method.

https://doi.org/10.1371/journal.pone.0136076.t007

Download:

Table 8. Best relative accuracy rate Γ_max obtained with the tiebreaker algorithm for the style identification task.

https://doi.org/10.1371/journal.pone.0136076.t008

Download:

Fig 9. Style identification.

Projection of the Brown dataset using topological features of networks modeling texts. The linear discriminant analysis [68] was employed to generate the visualization. Note that the variability of the documents classified as imaginative prose is lower than the variability of style observed for informative documents.

https://doi.org/10.1371/journal.pone.0136076.g009

Conclusions

In this study, I have shown that the styles of texts can be captured by measuring the topological properties of the corresponding network representation. More important than just to note a dependency between the structure of networks and stylistic features of texts, this study devised techniques to combine traditional and topological features, which were found to be of paramount importance to enhance the quality of current classification strategies. Two traditional stylometry tasks were studied: the authorship attribution and genre identification problems. In both tasks, the addition of topological features provided an improvement in classification performance. The highest improvement occurred with the hybrid classifier, which uses a linear convex combination of features from distinct textual evidences. The different nature of the quantities used to characterize texts suggests a complementary role in capturing distinct aspects of written texts. Because the adequate choice of parameters for the proposed technique is far from being a trivial task, I intend, as a future work, to devise a method that automatically assigns a value of λ and Θ for each test instance.

Supporting Information

S1 File. List of books employed in the experiments.

https://doi.org/10.1371/journal.pone.0136076.s001

(ZIP)

Author Contributions

Conceived and designed the experiments: DRA. Performed the experiments: DRA. Analyzed the data: DRA. Contributed reagents/materials/analysis tools: DRA. Wrote the paper: DRA.

References

1. Zipf GK. Human Behavior and the Principle of Least Effort. Addison-Wesley: Cambridge; 1949.
2. Kali R. The city as a giant component: a random graph approach to Zipf’s law. Appl Econ Lett. 2003; 10: 717–720.
- View Article
- Google Scholar
3. Baixeries J, Elvevag B, Ferrer-i-Cancho R. The evolution of the exponent of Zipf’s law in language ontogeny. PLoS ONE 2013; 8: e53227. pmid:23516390
- View Article
- PubMed/NCBI
- Google Scholar
4. Bochkarev VV, Lerner EY, Shevlyakova AV. Deviations in the zipf and heaps laws in natural languages. J Phys: Conf Ser. 2014; 490: 012009.
- View Article
- Google Scholar
5. Font-Clos F, Boleda G, Corral A. A scaling law beyond Zipf’s law and its relation to Heaps’ law. New J Phys. 2013; 15: 093033.
- View Article
- Google Scholar
6. Lu L, Zhang Z, Zhou T. Deviation of Zipf’s and Heaps’ laws in human languages with limited dictionary sizes. Sci Rep. 2013; 3: 1082. pmid:23378896
- View Article
- PubMed/NCBI
- Google Scholar
7. Katz SM. Distribution of content words and phrases in text and language modelling. Nat Lang Eng. 1996; 2: 15–59.
- View Article
- Google Scholar
8. Amancio DR. Authorship recognition via fluctuation analysis of network topology and word intermittency. J Stat Mech. 2015; P03005.
- View Article
- Google Scholar
9. Carpena P, Bernaola-Galvan P, Hackenberg M, Coronado AV, Oliver JL. Level statistics of words: finding keywords in literary texts and symbolic sequences. Phys Rev E. 2009; 79: 3.
- View Article
- Google Scholar
10. Herrera JP, Pury PA. Statistical keyword detection in literary corpora. EPJ B. 2008; 63: 135–146.
- View Article
- Google Scholar
11. Allegrini P, Grigolini P, Palatella L. Intermittency and scale-free networks: a dynamical model for human language complexity. Chaos Soliton Fract. 2004; 20: 95.
- View Article
- Google Scholar
12. Kulig A, Drozdz S, Kwapien J, Oswiecimka P. Modeling the average shortest-path length in growth of word-adjacency networks. Phys Rev E. 2015; 91: 032810.
- View Article
- Google Scholar
13. Roxas RM, Tapang G. Prose and poetry classification and boundary detection using word adjacency network analysis. Int J Mod Phys C. 2010; 21: 503–512.
- View Article
- Google Scholar
14. Masucci AP, Rodgers GJ. Network properties of written human language. Phys Rev E. 2006; 74: 026102.
- View Article
- Google Scholar
15. Cong J, Liu H. Approaching human language with complex networks. Phys Life Rev. 2014; 11: 4.
- View Article
- Google Scholar
16. Ferrer i Cancho R, Sole RV Least effort and the origins of scaling in human language. Proc Natl Acad Sci USA. 2003; 100: 788–91. pmid:12540826
- View Article
- PubMed/NCBI
- Google Scholar
17. Corominas-Murtra B, Balverde S, Solé R. The ontogeny of scale-free syntax networks: phase transitions in early language acquisition. Advs Complex Syst. 2009; 12: 371–392.
- View Article
- Google Scholar
18. Amancio DR, Antiqueira L, Pardo TAS, Oliveira ON Jr., Nunes MGV. Complex networks analysis of manual and machine translations. Int J Mod Phys C. 2008; 19: 583.
- View Article
- Google Scholar
19. Amancio DR, Nunes MGV, Oliveira ON Jr., Costa LF. Extractive summarization using complex networks and syntactic dependency. Physica A. 2012; 391: 1855–1864.
- View Article
- Google Scholar
20. Amancio DR, Oliveira ON Jr., Costa LF. Unveiling the relationship between complex networks metrics and word senses. Europhys Lett. 2012; 98: 18002.
- View Article
- Google Scholar
21. Navigli R. Word sense disambiguation: a survey. ACM Comput Surv. 2009: 41; 2.
- View Article
- Google Scholar
22. Amancio DR, Aluisio SM, Oliveira ON Jr., Costa LF. Complex networks analysis of language complexity. Europhys Lett. 2012; 100: 58002.
- View Article
- Google Scholar
23. Antiqueira L, Nunes MGV, Oliveira ON Jr, Costa LF. Strong correlations between text quality and complex networks features. Physica A. 2007; 373: 811–820.
- View Article
- Google Scholar
24. Mihalcea R, Radev D. Graph-based natural language processing and information retrieval. Cambridge University Press: Cambridge; 2012.
25. Stamatatos E. A survey of modern authorship attribution methods. J Am Soc Inf Sci Technol. 2009; 60: 538–556.
- View Article
- Google Scholar
26. Yu B. An Evaluation of Text Classification Methods for Literary Study. Ph.D. Dissertation. University of Illinois at Urbana-Champaign, Champaign, IL, USA. 2006.
27. Keller JM, Gray MR, Givens JA Jr. A fuzzy k-nearest neighbor algorithm. IEEE Trans Syst Man, Cybern. 1985; 15: 580–585.
- View Article
- Google Scholar
28. Baronchelli A, Ferrer-i-Cancho R, Pastor-Satorras R, Chater N, Christiansen MH. Networks in cognitive science. Trends Cogn Sci. 2013; 17: 348–60. pmid:23726319
- View Article
- PubMed/NCBI
- Google Scholar
29. Antiqueira L, Oliveira ON Jr., Costa LF, Nunes MGV. A complex network approach to text summarization. Inform Sci. 2009; 179: 584–599.
- View Article
- Google Scholar
30. Ferrer i Cancho R, Solé R, Kohler R. Patterns in syntactic dependency networks. Phys Rev E. 2004; 69: 51915.
- View Article
- Google Scholar
31. Capitan JA et al. Local-based semantic navigation on a networked representation of information. PLoS ONE. 2012; 7: e43694. pmid:22937081
- View Article
- PubMed/NCBI
- Google Scholar
32. Gravino P, Servedio VDP, Barrat A, Loreto V. Complex structures and semantics in free word association. Adv Complex Syst. 2012; 15: 1250054–1.
- View Article
- Google Scholar
33. Silva TC, Amancio DR. Word sense disambiguation via high order of learning in complex networks. Europhys Lett. 2012: 98; 58001.
- View Article
- Google Scholar
34. Veronis J. Hyperlex: lexical cartography for information retrieval. Comput Speech Lang. 2004; 18: 3 223–252.
- View Article
- Google Scholar
35. Widdows D, Dorow B. A graph model for unsupervised lexical acquisition. Proceedings of the 19th International Conference on Computational Linguistics. 2002; 1–7.
36. Martinez-Romo J et al. Disentangling categorical relationships through a graph of co-occurrences. Phys Rev E. 2011; 84: 046108.
- View Article
- Google Scholar
37. Amancio DR, Oliveira ON Jr., Costa LF. Identification of literary movements using complex networks to represent texts. New J Phys. 2012; 14: 043029.
- View Article
- Google Scholar
38. Amancio DR et al. Using metrics from complex networks to evaluate machine translation. Physica A. 2011; 390: 131–142.
- View Article
- Google Scholar
39. Amancio DR, Altmann EG, Rybski D, Oliveira ON Jr., Costa LF. Probing the statistical properties of unknown texts: application to the Voynich manuscript. PLoS ONE. 2013; 8: e67310. pmid:23844002
- View Article
- PubMed/NCBI
- Google Scholar
40. Berger AL, Della Pietra VJ, Della Pietra SA. A maximum entropy approach to natural language processing. Comput Linguist. 1996; 22: 1 39–71.
- View Article
- Google Scholar
41. Costa LF, Rodrigues FA, Travieso G, Villas Boas PRV. Characterization of complex networks: a survey of measurements. Adv Phys. 2007; 56: 167–242.
- View Article
- Google Scholar
42. Antiqueira L, Pardo TAS, Nunes MGV, Oliveira ON Jr. Some issues on complex networks for author characterization. Inteligencia Artificial. 2007: 11; 51–58.
- View Article
- Google Scholar
43. Travencolo B, Costa LF. Accessibility in complex networks. Phys Lett A. 2008; 373: 89–95.
- View Article
- Google Scholar
44. Costa LF, Tognetti MAR, Silva FN. Concentric characterization and classification of complex network nodes: application to an institutional collaboration network. Physica A. 2008; 387: 6201–6214.
- View Article
- Google Scholar
45. Amancio DR. Probing the topological properties of complex networks modeling short written texts. PLoS ONE. 2015; 10: e0118394. pmid:25719799
- View Article
- PubMed/NCBI
- Google Scholar
46. Fortunato S. Community detection in graphs. Phys Rep. 2010; 486: 75–174.
- View Article
- Google Scholar
47. Amancio DR, Altmann EG, Oliveira ON Jr., Costa LF. Comparing intermittency and network measurements of words and their dependence on authorship. New J Phys. 2011; 13: 123024.
- View Article
- Google Scholar
48. Newman MEJ. Mixing patterns in networks. Phys Rev E. 2003; 67: 026126.
- View Article
- Google Scholar
49. Manning CD, Schutze H. Foundations of Statistical Natural Language Processing. MIT Press: Cambridge; 1999.
50. Mosteller F, Wallace D. Inference and Disputed Authorship: The Federalist. Addison-Wesley: Reading; 1964.
51. Havlin S. The distance between zipf plots. Physica A. 1995; 216: 148–50.
- View Article
- Google Scholar
52. Vilensky B. Can analysis of word frequency distinguish between writings of different authors? Physica A. 1996; 231: 705–11.
- View Article
- Google Scholar
53. Yang ACC, Peng CK, Yien WK, Goldberger AL. Information categorization approach to literary authorship disputes. Physica A. 2003; 329: 473–83.
- View Article
- Google Scholar
54. Jankowska M, Keselij V, Milios E. Relative n-gram signatures: document visualization at the level of character n-grams. IEEE Conference on Visual Analytics Science and Technology, Seattle, WA, USA. 2012.
55. Goh KI, Barabási AL. Burstiness and memory in complex systems. Europhys Lett. 2008; 81: 48002.
- View Article
- Google Scholar
56. Witten IH, Frank E. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann: San Francisco; 2005.
57. Mitchell TM. Machine Learning. The Mc-Graw-Hill Companies; 1997.
58. Lin C-F, Wang S-D. Fuzzy support vector machines. IEEE Trans Neural Netw Learn Syst. 2002; 13: 464–471.
- View Article
- Google Scholar
59. Olaru C, Wehenkel L. A complete fuzzy decision tree technique. Fuzzy Set Syst. 2003; 138: 221–254.
- View Article
- Google Scholar
60. Bonissone P, Cadenas JM, Garrido MC, Daz-Valladares RA. A fuzzy random forest. Int J Approx Reason. 2010; 51: 729–747.
- View Article
- Google Scholar
61. Pal SK and Mitra S (1992) Multilayer perceptron, fuzzy sets, and classification. IEEE Trans. Neural Netw. Learn. Syst. 3(5) 683–697.
- View Article
- Google Scholar
62. Bergstra J, Bengio Y. Random search for hyper-parameter optimization. J Machine Learning Research. 2012; 13: 281–305.
- View Article
- Google Scholar
63. Bergstra J, Bardenet R, Bengio Y, Kegl B. Algorithms for hyper-parameter optimization. Adv Neural Inform Process Syst. 2011; 24: 2546–2554
- View Article
- Google Scholar
64. Amancio DR et al. A systematic comparison of supervised classifiers. PLoS ONE. 2014; 9: e94137. pmid:24763312
- View Article
- PubMed/NCBI
- Google Scholar
65. Oakes M. Ant colony optimisation for stylometry: the federalist papers. Proc. 5th Int. Conf. on Recent Advances in Soft Computing; 2004.
66. Patton JM, Can F. A stylometric analysis of Yasar Kemal’s Ince Memed tetralogy. Computers and the Humanities. 2004: 38; 457–467.
- View Article
- Google Scholar
67. Available from http://clu.uni.no/icame/manuals/BROWN/INDEX.HTM
68. McLachlan GJ. Discriminant Analysis and Statistical Pattern Recognition. Wiley Interscience; 2004.

[ref1] 1. Zipf GK. Human Behavior and the Principle of Least Effort. Addison-Wesley: Cambridge; 1949.

[ref2] 2. Kali R. The city as a giant component: a random graph approach to Zipf’s law. Appl Econ Lett. 2003; 10: 717–720.
View Article
Google Scholar

[3] View Article

[4] Google Scholar

[ref3] 3. Baixeries J, Elvevag B, Ferrer-i-Cancho R. The evolution of the exponent of Zipf’s law in language ontogeny. PLoS ONE 2013; 8: e53227. pmid:23516390
View Article
PubMed/NCBI
Google Scholar

[6] View Article

[7] PubMed/NCBI

[8] Google Scholar

[ref4] 4. Bochkarev VV, Lerner EY, Shevlyakova AV. Deviations in the zipf and heaps laws in natural languages. J Phys: Conf Ser. 2014; 490: 012009.
View Article
Google Scholar

[10] View Article

[11] Google Scholar

[ref5] 5. Font-Clos F, Boleda G, Corral A. A scaling law beyond Zipf’s law and its relation to Heaps’ law. New J Phys. 2013; 15: 093033.
View Article
Google Scholar

[13] View Article

[14] Google Scholar

[ref6] 6. Lu L, Zhang Z, Zhou T. Deviation of Zipf’s and Heaps’ laws in human languages with limited dictionary sizes. Sci Rep. 2013; 3: 1082. pmid:23378896
View Article
PubMed/NCBI
Google Scholar

[16] View Article

[17] PubMed/NCBI

[18] Google Scholar

[ref7] 7. Katz SM. Distribution of content words and phrases in text and language modelling. Nat Lang Eng. 1996; 2: 15–59.
View Article
Google Scholar

[20] View Article

[21] Google Scholar

[ref8] 8. Amancio DR. Authorship recognition via fluctuation analysis of network topology and word intermittency. J Stat Mech. 2015; P03005.
View Article
Google Scholar

[23] View Article

[24] Google Scholar

[ref9] 9. Carpena P, Bernaola-Galvan P, Hackenberg M, Coronado AV, Oliver JL. Level statistics of words: finding keywords in literary texts and symbolic sequences. Phys Rev E. 2009; 79: 3.
View Article
Google Scholar

[26] View Article

[27] Google Scholar

[ref10] 10. Herrera JP, Pury PA. Statistical keyword detection in literary corpora. EPJ B. 2008; 63: 135–146.
View Article
Google Scholar

[29] View Article

[30] Google Scholar

[ref11] 11. Allegrini P, Grigolini P, Palatella L. Intermittency and scale-free networks: a dynamical model for human language complexity. Chaos Soliton Fract. 2004; 20: 95.
View Article
Google Scholar

[32] View Article

[33] Google Scholar

[ref12] 12. Kulig A, Drozdz S, Kwapien J, Oswiecimka P. Modeling the average shortest-path length in growth of word-adjacency networks. Phys Rev E. 2015; 91: 032810.
View Article
Google Scholar

[35] View Article

[36] Google Scholar

[ref13] 13. Roxas RM, Tapang G. Prose and poetry classification and boundary detection using word adjacency network analysis. Int J Mod Phys C. 2010; 21: 503–512.
View Article
Google Scholar

[38] View Article

[39] Google Scholar

[ref14] 14. Masucci AP, Rodgers GJ. Network properties of written human language. Phys Rev E. 2006; 74: 026102.
View Article
Google Scholar

[41] View Article

[42] Google Scholar

[ref15] 15. Cong J, Liu H. Approaching human language with complex networks. Phys Life Rev. 2014; 11: 4.
View Article
Google Scholar

[44] View Article

[45] Google Scholar

[ref16] 16. Ferrer i Cancho R, Sole RV Least effort and the origins of scaling in human language. Proc Natl Acad Sci USA. 2003; 100: 788–91. pmid:12540826
View Article
PubMed/NCBI
Google Scholar

[47] View Article

[48] PubMed/NCBI

[49] Google Scholar

[ref17] 17. Corominas-Murtra B, Balverde S, Solé R. The ontogeny of scale-free syntax networks: phase transitions in early language acquisition. Advs Complex Syst. 2009; 12: 371–392.
View Article
Google Scholar

[51] View Article

[52] Google Scholar

[ref18] 18. Amancio DR, Antiqueira L, Pardo TAS, Oliveira ON Jr., Nunes MGV. Complex networks analysis of manual and machine translations. Int J Mod Phys C. 2008; 19: 583.
View Article
Google Scholar

[54] View Article

[55] Google Scholar

[ref19] 19. Amancio DR, Nunes MGV, Oliveira ON Jr., Costa LF. Extractive summarization using complex networks and syntactic dependency. Physica A. 2012; 391: 1855–1864.
View Article
Google Scholar

[57] View Article

[58] Google Scholar

[ref20] 20. Amancio DR, Oliveira ON Jr., Costa LF. Unveiling the relationship between complex networks metrics and word senses. Europhys Lett. 2012; 98: 18002.
View Article
Google Scholar

[60] View Article

[61] Google Scholar

[ref21] 21. Navigli R. Word sense disambiguation: a survey. ACM Comput Surv. 2009: 41; 2.
View Article
Google Scholar

[63] View Article

[64] Google Scholar

[ref22] 22. Amancio DR, Aluisio SM, Oliveira ON Jr., Costa LF. Complex networks analysis of language complexity. Europhys Lett. 2012; 100: 58002.
View Article
Google Scholar

[66] View Article

[67] Google Scholar

[ref23] 23. Antiqueira L, Nunes MGV, Oliveira ON Jr, Costa LF. Strong correlations between text quality and complex networks features. Physica A. 2007; 373: 811–820.
View Article
Google Scholar

[69] View Article

[70] Google Scholar

[ref24] 24. Mihalcea R, Radev D. Graph-based natural language processing and information retrieval. Cambridge University Press: Cambridge; 2012.

[ref25] 25. Stamatatos E. A survey of modern authorship attribution methods. J Am Soc Inf Sci Technol. 2009; 60: 538–556.
View Article
Google Scholar

[73] View Article

[74] Google Scholar

[ref26] 26. Yu B. An Evaluation of Text Classification Methods for Literary Study. Ph.D. Dissertation. University of Illinois at Urbana-Champaign, Champaign, IL, USA. 2006.

[ref27] 27. Keller JM, Gray MR, Givens JA Jr. A fuzzy k-nearest neighbor algorithm. IEEE Trans Syst Man, Cybern. 1985; 15: 580–585.
View Article
Google Scholar

[77] View Article

[78] Google Scholar

[ref28] 28. Baronchelli A, Ferrer-i-Cancho R, Pastor-Satorras R, Chater N, Christiansen MH. Networks in cognitive science. Trends Cogn Sci. 2013; 17: 348–60. pmid:23726319
View Article
PubMed/NCBI
Google Scholar

[80] View Article

[81] PubMed/NCBI

[82] Google Scholar

[ref29] 29. Antiqueira L, Oliveira ON Jr., Costa LF, Nunes MGV. A complex network approach to text summarization. Inform Sci. 2009; 179: 584–599.
View Article
Google Scholar

[84] View Article

[85] Google Scholar

[ref30] 30. Ferrer i Cancho R, Solé R, Kohler R. Patterns in syntactic dependency networks. Phys Rev E. 2004; 69: 51915.
View Article
Google Scholar

[87] View Article

[88] Google Scholar

[ref31] 31. Capitan JA et al. Local-based semantic navigation on a networked representation of information. PLoS ONE. 2012; 7: e43694. pmid:22937081
View Article
PubMed/NCBI
Google Scholar

[90] View Article

[91] PubMed/NCBI

[92] Google Scholar

[ref32] 32. Gravino P, Servedio VDP, Barrat A, Loreto V. Complex structures and semantics in free word association. Adv Complex Syst. 2012; 15: 1250054–1.
View Article
Google Scholar

[94] View Article

[95] Google Scholar

[ref33] 33. Silva TC, Amancio DR. Word sense disambiguation via high order of learning in complex networks. Europhys Lett. 2012: 98; 58001.
View Article
Google Scholar

[97] View Article

[98] Google Scholar

[ref34] 34. Veronis J. Hyperlex: lexical cartography for information retrieval. Comput Speech Lang. 2004; 18: 3 223–252.
View Article
Google Scholar

[100] View Article

[101] Google Scholar

[ref35] 35. Widdows D, Dorow B. A graph model for unsupervised lexical acquisition. Proceedings of the 19th International Conference on Computational Linguistics. 2002; 1–7.

[ref36] 36. Martinez-Romo J et al. Disentangling categorical relationships through a graph of co-occurrences. Phys Rev E. 2011; 84: 046108.
View Article
Google Scholar

[104] View Article

[105] Google Scholar

[ref37] 37. Amancio DR, Oliveira ON Jr., Costa LF. Identification of literary movements using complex networks to represent texts. New J Phys. 2012; 14: 043029.
View Article
Google Scholar

[107] View Article

[108] Google Scholar

[ref38] 38. Amancio DR et al. Using metrics from complex networks to evaluate machine translation. Physica A. 2011; 390: 131–142.
View Article
Google Scholar

[110] View Article

[111] Google Scholar

[ref39] 39. Amancio DR, Altmann EG, Rybski D, Oliveira ON Jr., Costa LF. Probing the statistical properties of unknown texts: application to the Voynich manuscript. PLoS ONE. 2013; 8: e67310. pmid:23844002
View Article
PubMed/NCBI
Google Scholar

[113] View Article

[114] PubMed/NCBI

[115] Google Scholar

[ref40] 40. Berger AL, Della Pietra VJ, Della Pietra SA. A maximum entropy approach to natural language processing. Comput Linguist. 1996; 22: 1 39–71.
View Article
Google Scholar

[117] View Article

[118] Google Scholar

[ref41] 41. Costa LF, Rodrigues FA, Travieso G, Villas Boas PRV. Characterization of complex networks: a survey of measurements. Adv Phys. 2007; 56: 167–242.
View Article
Google Scholar

[120] View Article

[121] Google Scholar

[ref42] 42. Antiqueira L, Pardo TAS, Nunes MGV, Oliveira ON Jr. Some issues on complex networks for author characterization. Inteligencia Artificial. 2007: 11; 51–58.
View Article
Google Scholar

[123] View Article

[124] Google Scholar

[ref43] 43. Travencolo B, Costa LF. Accessibility in complex networks. Phys Lett A. 2008; 373: 89–95.
View Article
Google Scholar

[126] View Article

[127] Google Scholar

[ref44] 44. Costa LF, Tognetti MAR, Silva FN. Concentric characterization and classification of complex network nodes: application to an institutional collaboration network. Physica A. 2008; 387: 6201–6214.
View Article
Google Scholar

[129] View Article

[130] Google Scholar

[ref45] 45. Amancio DR. Probing the topological properties of complex networks modeling short written texts. PLoS ONE. 2015; 10: e0118394. pmid:25719799
View Article
PubMed/NCBI
Google Scholar

[132] View Article

[133] PubMed/NCBI

[134] Google Scholar

[ref46] 46. Fortunato S. Community detection in graphs. Phys Rep. 2010; 486: 75–174.
View Article
Google Scholar

[136] View Article

[137] Google Scholar

[ref47] 47. Amancio DR, Altmann EG, Oliveira ON Jr., Costa LF. Comparing intermittency and network measurements of words and their dependence on authorship. New J Phys. 2011; 13: 123024.
View Article
Google Scholar

[139] View Article

[140] Google Scholar

[ref48] 48. Newman MEJ. Mixing patterns in networks. Phys Rev E. 2003; 67: 026126.
View Article
Google Scholar

[142] View Article

[143] Google Scholar

[ref49] 49. Manning CD, Schutze H. Foundations of Statistical Natural Language Processing. MIT Press: Cambridge; 1999.

[ref50] 50. Mosteller F, Wallace D. Inference and Disputed Authorship: The Federalist. Addison-Wesley: Reading; 1964.

[ref51] 51. Havlin S. The distance between zipf plots. Physica A. 1995; 216: 148–50.
View Article
Google Scholar

[147] View Article

[148] Google Scholar

[ref52] 52. Vilensky B. Can analysis of word frequency distinguish between writings of different authors? Physica A. 1996; 231: 705–11.
View Article
Google Scholar

[150] View Article

[151] Google Scholar

[ref53] 53. Yang ACC, Peng CK, Yien WK, Goldberger AL. Information categorization approach to literary authorship disputes. Physica A. 2003; 329: 473–83.
View Article
Google Scholar

[153] View Article

[154] Google Scholar

[ref54] 54. Jankowska M, Keselij V, Milios E. Relative n-gram signatures: document visualization at the level of character n-grams. IEEE Conference on Visual Analytics Science and Technology, Seattle, WA, USA. 2012.

[ref55] 55. Goh KI, Barabási AL. Burstiness and memory in complex systems. Europhys Lett. 2008; 81: 48002.
View Article
Google Scholar

[157] View Article

[158] Google Scholar

[ref56] 56. Witten IH, Frank E. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann: San Francisco; 2005.

[ref57] 57. Mitchell TM. Machine Learning. The Mc-Graw-Hill Companies; 1997.

[ref58] 58. Lin C-F, Wang S-D. Fuzzy support vector machines. IEEE Trans Neural Netw Learn Syst. 2002; 13: 464–471.
View Article
Google Scholar

[162] View Article

[163] Google Scholar

[ref59] 59. Olaru C, Wehenkel L. A complete fuzzy decision tree technique. Fuzzy Set Syst. 2003; 138: 221–254.
View Article
Google Scholar

[165] View Article

[166] Google Scholar

[ref60] 60. Bonissone P, Cadenas JM, Garrido MC, Daz-Valladares RA. A fuzzy random forest. Int J Approx Reason. 2010; 51: 729–747.
View Article
Google Scholar

[168] View Article

[169] Google Scholar

[ref61] 61. Pal SK and Mitra S (1992) Multilayer perceptron, fuzzy sets, and classification. IEEE Trans. Neural Netw. Learn. Syst. 3(5) 683–697.
View Article
Google Scholar

[171] View Article

[172] Google Scholar

[ref62] 62. Bergstra J, Bengio Y. Random search for hyper-parameter optimization. J Machine Learning Research. 2012; 13: 281–305.
View Article
Google Scholar

[174] View Article

[175] Google Scholar

[ref63] 63. Bergstra J, Bardenet R, Bengio Y, Kegl B. Algorithms for hyper-parameter optimization. Adv Neural Inform Process Syst. 2011; 24: 2546–2554
View Article
Google Scholar

[177] View Article

[178] Google Scholar

[ref64] 64. Amancio DR et al. A systematic comparison of supervised classifiers. PLoS ONE. 2014; 9: e94137. pmid:24763312
View Article
PubMed/NCBI
Google Scholar

[180] View Article

[181] PubMed/NCBI

[182] Google Scholar

[ref65] 65. Oakes M. Ant colony optimisation for stylometry: the federalist papers. Proc. 5th Int. Conf. on Recent Advances in Soft Computing; 2004.

[ref66] 66. Patton JM, Can F. A stylometric analysis of Yasar Kemal’s Ince Memed tetralogy. Computers and the Humanities. 2004: 38; 457–467.
View Article
Google Scholar

[185] View Article

[186] Google Scholar

[ref67] 67. Available from http://clu.uni.no/icame/manuals/BROWN/INDEX.HTM

[ref68] 68. McLachlan GJ. Discriminant Analysis and Statistical Pattern Recognition. Wiley Interscience; 2004.

Figures

Abstract

Introduction

Representing texts as networks

Topological characterization of networks

Traditional stylistic features

Frequency of words and characters

Intermittence

Supervised classification

Hybrid classifier

Tiebreaker classifier

Results and Discussion

Authorship recognition

Style identification

Conclusions

Supporting Information

S1 File. List of books employed in the experiments.

Author Contributions

References