An existing classification: The USPC system

Each USPTO patent is associated with a non-empty set of technological classes and subclasses. There are currently around 440 classes and over 150,000 subclasses constituting the United State Patent Classification (USPC) system. While a technological class corresponds to the technological field covered by the patent, a subclass stands for a specific technology or method used in this invention. A patent can have multiple technological classes, on average in our data a patent has 1.8 different classes and 3.9 pairs of class/subclass. At this stage, two features of this system are worth mentioning: (i) classes and subclasses are not chosen by the inventors of the patent but by the examiner during the granting process based on the content of the patent; (ii) the classification has evolved in time and continues to change in order to adapt to new technologies by creating or editing classes. When a change occurs, the USPTO reviews all the previous patents so as to create a consistent classification.

A bibliographical network between patents: Citations

As with scientific publications, patents must give reference to all the previous patents which correspond to related prior art. They therefore indicate the past knowledge which relates to the patented invention. Yet, contrary to scientific citations, they also have an important legal role as they are used to delimit the scope of the property rights awarded by the patent. One can consult [24] for more details about this. Failing to refer to prior art can lead to the invalidation of the patent (e.g. [25]). Another crucial difference is that the majority of the citations are actually chosen by the examiners and not by the inventors themselves. From the USPTO, we gather information of all citations made by each patent (backward citations) and all citations received by each patent as of the end of 2013 (forward citations). We can thus build a complete network of citations that we will use later on in the analysis.

Turning to the structure of the lag between the citing and the cited patent in terms of application date, we see that the mean of this lag is 8.5 years and the median is 7 years. This distribution is highly skewed, the 95^th percentile is 21 years. We also report 164,000 citations with a negative time lag. This is due to the fact that some citations can be added during the examination process and some patents require more time to be granted than others.

In what follows, we choose to restrict attention to pairs of citations with a lag no larger than 5 years. We impose this restriction for two reasons. First, the number of citations received peaks 4–5 years after application. Second, the structure of the citation lag is necessarily biased by the truncation of our sample: the more recent patents mechanically receive less citations than the older ones. As we are restricting to citations received no later than 5 years after the application date, this effect will only affect patents with an application date after 2007.

Data collection and basic description

Each patent contains an abstract and a core text which describe the invention. To see what a patent looks like in practice, one can refer to the USPTO patent full-text database http://patft.uspto.gov/netahtml/PTO/index.html or to Google patent which publishes USPTO patents in pdf format at https://patents.google.com. Although including the full core texts would be natural and probably very useful in a systematic text-mining approach as done in [26], they are too long to be included and thus we consider only the abstracts for the analysis. Indeed, the semantic analysis counts more than 4 million patents, with corresponding abstracts with an average length of 120.8 words (and a standard deviation of 62.4), a size that is already challenging in terms of computational burden and data size. In addition, abstracts are aimed at synthesizing purpose and content of patents and must therefore be a relevant object of study (see [27]). The USPTO defines a guidance stating that an abstract should be “a summary of the disclosure as contained in the description, the claims, and any drawings; the summary shall indicate the technical field to which the invention pertains and shall be drafted in a way which allows the clear understanding of the technical problem, the gist of the solution of that problem through the invention, and the principal use or uses of the invention” (PCT Rule 8).

We construct from raw data a unified database. Data is collected from USPTO patent redbook bulk downloads, that provides as raw data (specific dat or xml formats) full patent information, starting from 1976. Detailed procedure of data collection, parsing and consolidation are available in S2 Text. The latest dump of the database in Mongodb format is available at http://dx.doi.org/10.7910/DVN/BW3ACK. Collection and homogenization of the database into a directly usable database with basic information and abstracts was an important task as USPTO raw data formats are involved and change frequently.

We count 4,666,365 utility patents with an abstract granted from 1976 to 2013. A very small number of patents have a missing abstract, these are patents that have been withdrawn and we do not consider them in the analysis. The number of patents granted each year increases from around 70,000 in 1976 to about 278,000 in 2013. When distributed by the year of application, the picture is slightly different. The number of patents steadily increase from 1976 to 2000 and remains constant around 200,000 per year from 2000 to 2007. Restricting our sample to patent with application date ranging from 1976 to 2007, we are left with 3,949,615 patents. These patents cite 38,756,292 other patents with the empirical lag distribution that has been extensively analyzed in [4]. Conditioned on being cited at least once, a patent receives on average 13.5 citations within a five-year window. 270,877 patents receive no citation during the next five years following application, 10% of patents receive only one citation and 1% of them receive more than 100 citations. A within class citation is defined as a citation between two patents sharing at least one common technological class. Following this definition, 84% of the citations are within class citations. 14% of the citations are between two patents that share the exact same set of technological classes.

Towards a complementary classification

Potentialities of text-mining techniques as an alternative way to analyze and classify patents are documented in [26]. The author’s main argument, in support of an automatic classification tool for patent, is to reduce the considerable amount of human effort needed to classify all the applications. The work conducted in the field of natural language processing and/or text analysis has been developed in order to improve search performance in patent databases, build technology map or investigate the potential infringement risks prior to developing a new technology (see [28] for a review). Text-mining of patent documents is also widely used as a tool to build networks which carry additional information to the simplistic bibliographic connections model as argued in [16]. As far as the authors know, the use of text-mining as a way to build a global classification of patents remains however largely unexplored. One notable exception can be found in [15] where semantic-based classification is shown to outperform the standard classification in predicting the convergence of technologies even in small samples. Semantic analysis reveals itself to be more flexible and more quickly adaptable to the apparition of new clusters of technologies. Indeed, as argued in [15], before two distinct technologies start to clearly converge, one should expect similar words to be used in patents from both technologies.

Finally, a semantic classification where patents are gathered based on the fact that they share similar significant keywords has the advantage of including a network feature that cannot be found in the USPC case, namely that each patent is associated with a vector of probability to belong to each of the semantic classes (more details on this feature can be found in Section 1). Using co-occurrence of keywords, it is then possible to construct a network of patents and to study the influence of some key topological features. As reviewed previously, the use of co-occurrences is the usual way to construct a semantic network. Other hybrid technique such as bipartite semantic/authors networks, do not have the nice feature of relying solely on endogenous semantic information contained in data.

Keywords extraction

Let be the set of patents, we first assign to a patent a set of potentially significant keywords K(p) from its text (that corresponds to the concatenation of its own title and abstract). K(p) are extracted through a similar procedure as the one detailed in [29]:

1. Text parsing and Tokenization: we transform raw texts into a set of words and sentences, reading it (parsing) and splitting it into elementary entities (words organized in sentences).
2. Part-of-speech tagging: attribution of a grammatical function to each of the tokens defined previously.
3. Stem extraction: families of words are generally derived from a unique root called stem (for example compute, computer, computation all yield the same stem comput) that we extract from tokens. At this point the abstract text is reduced to a set of stems and their grammatical functions.
4. Multi-stems construction: these are the basic semantic units used in further analysis. They are constructed as groups of successive stems in a sentence which satisfies a simple grammatical function rule. The length of the group is between 1 and 3 and its elements are either nouns, attributive verbs or adjectives. We choose to extract the semantics from such nominal groups in view of the technical nature of texts, which is not likely to contain subtle nuances in combinations of verbs and nominal groups.

Text processing operations are implemented in python in order to use built-in functions nltk library [30] for most of above operations. This library supports most of state-of-the-art natural language processing operations. Source code is openly available on the repository of the project at https://github.com/JusteRaimbault/PatentsMining.

Keywords relevance estimation

Construction of the semantic network

We keep the set of most relevant keywords and obtain their co-occurrence matrix as defined in Section 1. This matrix can be directly interpreted as the weighted adjacency matrix of the semantic network. At this stage, the topology of raw networks does not allow the extraction of clear communities. This is partly due to the presence of hubs that correspond to frequent terms common to many fields (e.g. method, apparat) which are wrongly filtered as relevant. We therefore introduce an additional measure to correct the network topology: the concentration of keywords across technological classes, defined as: where k_j(s) is the number of occurrences of the sth keyword in each of the jth technological class taken from one of the N^(tec) USPC classes. The higher c_tech, the more specific to a technological class the node is. For example, the terms semiconductor is widely used in electronics and does not contain any significant information in this field. We use a threshold parameter, defined as θ_c, and keep nodes with c_tech(s) > θ_c. Likewise, edges with low weights correspond to rare co-occurrences and are considered to be noise. To account for this we define the threshold parameter for edges θ_w, and we filter edges with a weight below θ_w, following the rationale that two keywords are not linked “by chance” if they appear simultaneously a minimal number of time. To control for size effect, we normalize by taking where N_P is the number of patents in the corpus (). is thus a varying parameter interpreted as a noise threshold per patent. Communities are then extracted using a standard modularity maximization procedure as described in [31] to which we add the two constraints captured by θ_w and θ_c, namely that edges must have a weight greater than θ_w and nodes a concentration greater than θ_c. At this stage, both parameters θ_c and are unconstrained and their choice is not straightforward. Indeed, many optimization objectives are possible, such as the modularity, network size or number of communities. We find that modularity is maximized at a roughly stable value of θ_w across different θ_c for each year, corresponding to a stable across years, which leads us to choose . Then for the choice of θ_c, different candidates points lie on a Pareto front for the bi-objective optimization on number of communities and network size. There is a priori no reason to choose any specific point among the different optimums. Consequently, we have tried the analysis with all the candidate values for θ_c and found that the results are the most reasonable when taking θ_c = 0.06 (see Fig 1). We show in Fig 2 an example of semantic network visualization.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Sensitivity analysis of network community structure to filtering parameters.

We consider a specific window 2000–2004 and the obtained plots are typical. (Left panel) We plot the number of communities as a function of the edge threshold parameter θ_w for different values of the node threshold parameter θ_c. The maximum is roughly stable across θ_c (dashed red line). (Right panel) To choose θ_c, we do a Pareto optimization on communities and network size: the compromise point (red overline) on the Pareto front (purple overline: possible choices after having fixed ; blue level gives modularity) corresponds to θ_c = 0.06.

https://doi.org/10.1371/journal.pone.0176310.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. An example of semantic network visualization.

We show the network obtained for the window 2000–2004, with parameters θ_c = 0.06 and . The corresponding file in a vector format (.svg), that can be zoomed and explored, is available as S1 File.

https://doi.org/10.1371/journal.pone.0176310.g002

Characteristics of semantic classes

For each year t, we define as the number of semantic classes which have been computed by clustering keywords from patents appeared during the period [t − T₀, t] (we recall that we have chosen T₀ = 4). Each semantic class is characterized by a set of keywords K(k, t) which is a subset of selected as described in previous sections. The cardinal of K(k, t) distribution across each semantic class k is highly skewed with a few semantic classes containing over 1,000 keywords, most of them with roughly the same number of keywords. In contrast, there are also many semantic classes with only two keywords. There are around 30 keywords by semantic class on average and the median is 2 for any t. Fig 3 shows that the average number of keywords is relatively stable from 1976 to 1992 and then picks around 1996 prior to going down.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. This figure plots the average number of keywords by semantic class for each time window [t − 4; t] from t = 1980 to t = 2007.

https://doi.org/10.1371/journal.pone.0176310.g003

Size of technological and semantic classes.

We consider a specific window of observations (for example 2000–2004), and we define Z the number of patents which appeared during that time window. For each patent i = 1, ⋯, Z we associate a vector of probability where each component , with j = 1, ⋯, N(sem) and where (when there is no room for confusion, we drop the subscript t in ). On average across all time windows, a patent is associated to 1.8 semantic classes with a positive probability. Next we define the size of a semantic class as Correspondingly, we aim to provide a consistent definition for technological classes. For that purpose, we follow the so-called “fractional count” method, which was introduced by the USPTO and consists in dividing equally the patents between all the classes they belong to. Formally, we define the number of technological classes as N^(tec) (which is not time dependent contrary to the semantic case) and for j = 1, ⋯, N^(tec) the corresponding matrix of probability is defined as where B_ij equals 1 if the ith patent belongs to the jth technological class and 0 if not. When there is no room for confusion, we will drop the exponent part and write only p_ij when referring to either the technological or semantic matrix. Empirically, we find that both classes exhibit a similar hierarchical structure in the sense of a power-law type of distribution of class sizes as shown in Fig 4. This feature is important, it suggests that a classification based on the text content of patents has some separating power in the sense that it does not divide up all the patents in one or two communities.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Sizes of classes.

Yearly from t = 1980 to t = 2007, we plot the size of semantic classes (left-side) and technological classes (right-side) for the corresponding time window [t − 4, t], from the biggest to the smallest. The formal definition of size can be found in Section Characteristics of Semantic Classes. Each color corresponds to one specific year. Yearly semantic classes and technological classes present a similar hierarchical structure which confirms the comparability of the two classifications. This feature is crucial for the statistical analysis in Section Statistical Model. Over time, curves are translated and levels of hierarchy stays roughly constant.

https://doi.org/10.1371/journal.pone.0176310.g004

Potential refinements of the method

Our semantic classification method could be refined by combining it with other techniques such as Latent Dirichlet Allocation which is a widely used topic detection method (e.g. [33]), already used on patent data as in [34] where it provides a measure of idea novelty and the counter-intuitive stylized facts that breakthrough invention are likely to come out of local search in a field rather than distant technological recombination. Using this approach should first help further evaluate the robustness of our qualitative conclusions (external validation). Also, depending on the level of orthogonality with our classification, it can potentially bring an additional feature to characterize patents, in the spirit of multi-modeling techniques where neighbor models are combined to take advantage of each point of view on a system.

Our use of network analysis can also be extended using newly developed techniques of hyper-network analysis. Indeed, patents and keywords can for example be nodes of a bipartite network, or patents be links of an hyper-network, in the sense of multiple layers with different classification links and citation links. The combination of citation network modeling by Stochastic Block Modeling with topic modeling was studied for scientific papers by [35], outperforming previous link prediction algorithms. [36] provide a method to compare macroscopic structures of the different layers in a multilayer network that could be applied as a refinement of the overlap, modularity and statistical modeling studied in this paper. Furthermore, is has recently been shown that measures of multilayer network projections induce a significant loss of information compared to the generalized corresponding measure [37], which confirms the relevance of such development that we left for further research.

An other potential research development would be to further exploit the temporal structure of our dataset. Indeed, large progress have recently been made in complex network analysis of time-series data (see [38] for a review). For example, [39] develops a method to construct multiscale network from time series, which could in our case be a solution to identify structures in patents trajectories at different levels, and be an alternative to the single scale modularity analysis we use.

Patent level measures

Given a classification system (technological or semantic classes), and the associated probabilities p_ij for each patent i to belong to class j (that were defined in Section Characteristics of Semantic Classes), one can define a patent-level diversity measure as one minus the Herfindhal concentration index on p_ij by

We show in Fig 5 the distribution over time of semantic and technological diversity with the corresponding mean time-series. This is carried with two different settings, namely including/not including patents with zero diversity (i.e. single class patents). We call other patents “complicated patents” in the following. First of all, the presence of mass in small probabilities for semantic but not technological diversity confirms that the semantic classification contains patent spread over a larger number of classes. More interestingly, a general decrease of diversity for complicated patents, both for semantic and technological classification systems, can be interpreted as an increase in invention specialization. This is a well-known stylized fact as documented in [40]. Furthermore, a qualitative regime shift on semantic classification occurs around 1996. This can be seen whether or not we include patents with zero diversity. The diversity of complicated patents stabilizes after a constant decrease, and the overall diversity begins to strongly decrease. This means that on the one hand the number of single class patents begins to increase and on the other hand complicated patents do not change in diversity. It can be interpreted as a change in the regime of specialization, the new regime being caused by more single-class patents.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Patent level diversities.

Distributions of diversities (Left column) and corresponding mean time-series (Right column) for t = 1980 to t = 2007 (with the corresponding time window [t − 4, t]). The first row includes all classified patents, whereas the second row includes only patents with more than one class (i.e. patents with diversity greater than 0).

https://doi.org/10.1371/journal.pone.0176310.g005

More commonly used in the literature are the measures of originality and generality. These measures follow the same idea than the above-defined diversity in quantifying the diversity of classes (whether technological or semantic) associated with a patent. But instead of looking at the patent’s classes, they consider the classes of the patents that are cited or citing. Formally, the originality O_i and the generality G_i of a patent i are defined as where z ∈ {tec, sem}, I_i denotes the set of patents that are cited by the ith patent within a five year window (i.e. if the ith patent appears at year t, then we consider patents on [t − T₀, t]) when considering the originality and the set of patents that cite patent i after less than five years (i.e. we consider patents on [t, t+T₀]) in the case of generality. Note that the measure of generality is forward looking in the sense that used information that will only be available 5 years after patent applications. Both measures are lower on average based on semantic classification than on technological classification. Fig 6 plots the mean value of , , and .

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Patent level originality (left hand side) and generality (right hand side) for t = 1980 to t = 2007 (with the corresponding time window [t − 4, t]) as defined in subsection Patent Level Measures.

https://doi.org/10.1371/journal.pone.0176310.g006

Classes overlaps

A proximity measure between two classes can be defined by their overlap in terms of patents. Such measures could for example be used to construct a metrics between semantic classes. Intuitively, highly overlapping classes are very close in terms of technological content and one can use them to measure distance between two firms in terms of technology as done in [41]. Formally, recalling the definition of (p_ij) as the probability for the ith patent to belong to the jth class and N_P as the number of patents it writes (2)

The overlap is normalized by patent count to account for the effect of corpus size: by convention, we assume the overlap to be maximal when there is only one class in the corpus. A corresponding relative overlap is computed as a set similarity measure in the number of patents common to two classes A and B, given by .

Intra-classification overlaps.

The study of distributions of overlaps inside each classification, i.e. between technological classes and between semantic classes separately, reveals the structural difference between the two classification methods, suggesting their complementary nature. Their evolution in time can furthermore give insights into trends of specialization. We show in Fig 7 distributions and mean time-series of overlaps for the two classifications. The technological classification globally always follow a decreasing trend, corresponding to more and more isolated classes, i.e. specialized inventions, confirming the stylized fact obtained in previous subsection. For semantic classes, the dynamic is somehow more intriguing and supports the story of a qualitative regime shift suggested before. Although globally decreasing as technological overlap, normalized (resp. relative) mean overlap exhibits a peak (clearer for normalized overlap) culminating in 1996 (resp. 1999). Looking at normalized overlaps, classification structure was somewhat stable until 1990, then strongly increased to peak in 1996 and then decrease at a similar pace up to now. Technologies began to share more and more until a breakpoint when increasing isolation became the rule again. An evolutionary perspective on technological innovation [42] could shed light on possible interpretations of this regime shift: as species evolve, the fitness landscape first would have been locally favorable to cross-insemination, until each fitness reaches a threshold above which auto-specialization becomes the optimal path. It is very comparable to the establishment of an ecological niche [43], the strong interdependency originating here during the mutual insemination resulting in a highly path-dependent final situation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Intra-classification overlaps.

(Left column) Distribution of overlaps O_ij for all i ≠ j (zero values are removed because of the log-scale). Right column) Corresponding mean time-series. (First row) Normalized overlaps. (Second row) Relative overlaps.

https://doi.org/10.1371/journal.pone.0176310.g007

Inter-classification overlaps.

Overlaps between classifications are defined as in (Classes overlaps), but with j standing for the jth technological class and k for the kth semantic class: p_ij are technological probabilities and p_ik semantic probabilities. They describe the relative correspondence between the two classifications and are a good indicator to spot relative changes, as shown in Fig 8. Mean inter-classification overlap clearly exhibits two linear trends, the first one being constant from 1980 to 1996, followed by a constant decrease. Although difficult to interpret directly, this stylized fact clearly unveils a change in the nature of inventions, or at least in the relation between content of inventions and technological classification. As the tipping point is at the same time as the ones observed in the previous section and since the two statistics are different, it is unlikely that this is a mere coincidence. Thus, these observations could be markers of a hidden underlying structural changes in processes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Distribution of relative overlaps between classifications.

(Left) Distribution of overlaps at all time steps; (Right) Corresponding mean time-series. The decreasing trend starting around 1996 confirms a qualitative regime shift in that period.

https://doi.org/10.1371/journal.pone.0176310.g008

Citation modularity

An exogenous source of information on relevance of classifications is the citation network described in Section A bibliographical network between patents: citations. The correspondence between citation links and classes should provide a measure of accuracy of classifications, in the sense of an external validation since it is well-known that citation homophily is expected to be quite high (see, e.g, [14]). This section studies empirically modularities of the citation network regarding the different classifications. To corroborate the obtained results, we propose to look at a more rigorous framework in Section Statistical Model. Modularity is a simple measure of how communities in a network are well clustered (see [31] for the accurate definition). Although initially designed for single-class classifications, this measure can be extended to the case where nodes can belong to several classes at the same time, in our case with different probabilities as introduced in [44]. The simple directed modularity is given in our case by with A_ij the citation adjacency matrix (i.e. A_ij = 1 if there is a citation from the ith patent to the jth patent, and A_ij = 0 if not), (resp. ) in-degree (resp. out-degree) of patents (i.e. the number of citations made by the ith patent to others and the number of citations received by the ith patent). Q_d can be defined for each of the two classification systems: z ∈ {tec, sem}. If z = tec, c_i is defined as the main patent class, which is taken as the first class whereas if z = sem, c_i is the class with the largest probability.

Multi-class modularity in turns is given by where We take F(p_ic, p_jc) = p_ic ⋅ p_jc as suggested in [44]. Modularity is an aggregated measure of how the network deviates from a null model where links would be randomly made according to node degree. In other words it captures the propensity for links to be inside the classes. Overlapping modularity naturally extends simple modularity by taking into account the fact that nodes can belong simultaneously to many classes. We document in Fig 9 both simple and multi-class modularities over time. For simple modularity, is low and stable across the years whereas is slightly greater and increasing. These values are however low and suggest that single classes are not sufficient to capture citation homophily. Multi-class modularities tell a different story. First of all, both classification modularities have a clear increasing trend, meaning that they become more and more adequate with citation network. The specializations revealed by both patent level diversities and classes overlap is a candidate explanation for this growing modularities. Secondly, semantic modularity dominates technological modularity by an order of magnitude (e.g. 0.0094 for technological against 0.0853 for semantic in 2007) at each time. This discrepancy has a strong qualitative significance. Our semantic classification fits better the citation network when using multiple classes. As technologies can be seen as a combination of different components as shown by [5], this heterogeneous nature is most likely better taken into account by our multi-class semantic classification.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Temporal evolution of semantic and technological modularities of the citation network.

(Left) Simple directed modularity, computed with patent main classes (main technological class and semantic class with larger probability). (Right) Multi-class modularity, computed following [44].

https://doi.org/10.1371/journal.pone.0176310.g009

Statistical model

In this section, we develop a statistical model aimed at quantifying performance of both technological and semantic classification systems. In particular, we aim at corroborating findings obtained in Section Citation Modularity. The mere difference between this approach and the citation modularity approach lies in the choice of the underlying model, and the according quantities of interest. In addition for the semantic approach, we want to see if when restricting to patents with higher probabilities to belong to a class, we obtain better results. To do that, we choose to look at within class citations proportion (for both technological and semantic approaches). We provide two obvious reasons why we choose this. First, the citations are commonly used as a proxy for performance as mentioned in Section Citation Modularity. Second, this choice is “statistically fair” in the sense that both approaches have focused on various goals and not on maximizing directly the within class proportion. Nonetheless, the within class proportion is too sensitive to the distribution of the shape of classes. For example, a dataset where patents for each class account for 10% of the total number of patents will mechanically have a better within class proportion than if each class accounts for only 1%. Consequently, an adequate statistical model, which treats datasets fairly regardless of their distribution in classes, is needed. This effort ressembles to the previous study of citation modularity, but is complementary since the model presented here can be understood as an elementary model of citation network growth. Furthermore, the parameters fitted here can have a direct interpretation as a citation probability.

We need to introduce and recall some notations. We consider a specific window of observations [t − T₀, t], and we define Z the number of patents which appeared during that time window. We let t₁, ⋯, t_Z their corresponding appearance date by chronological order, which for simplicity are assumed to be such that t₁ < ⋯ < t_Z. For each patent i = 1, ⋯, Z we consider C_i the number of distinctive couples {cited patent, cited patent’s class} made by the ith patent (for instance if the ith patent has only made one citation and that the cited patent is associated with three classes, then C_i = 3). Let z ∈ {tec, sem}, we define the number of patents associated to at least one of the ith classes at time t_{i − 1}. For l = 1, ⋯, C_i we consider the variables B_{l, i}, which equal 1 if the cited patent’s class is also common to the ith patent. We assume that B_{l, i} are independent of each other and conditioned on the past follow Bernoulli variables where the parameter 0 ≤ θ^(z) ≤ 1 indicates the propensity for any patent to cite patents of its own technological or semantic class. When θ^(z) = 0, the probability of citing patents from its own class is simply , which corresponds to the observed proportion of patents which belong to at least one of the ith patent’s classes. Thus this corresponds to the estimated probability of citing one patent if we assume that the probability of citing any patent k = 1, ⋯, i − 1 is uniformly distributed, which could be a reasonable assumption if classes were assigned randomly and independently from patent abstract contents. Conversely if θ^(z) = 1, we are in the case of a model where there are 100% of within class citations. A reasonable choice of θ^(z) lies between those two extreme values. Finally, we assume that the number of distinctive couples C_i are a sequence of independent and identically distributed random variables following the discrete distribution C, and also independent from the other quantities.

We estimate θ^(z) via maximum likelihood, and obtain the corresponding maximum likelihood estimator (MLE) . The likelihood function, along with the standard deviation expression and details about the test, can be found in S4 Text. The fitted values, standard errors and p-values corresponding to the statistical test θ^(sem) = θ^(tec) (with corresponding alternative hypothesis θ^(sem) > θ^(tec)) on non-overlapping blocks from the period 1980–2007 are reported on Table 1. Note that the estimation included patents up until 2010 in the period 2006–2007 and not the patents from 1980 in the period 1980–1985 for homogeneity in size with other periods. This doesn’t affect the significativity of the results. Semantic values are reported for four different chosen thresholds p⁻ = .04, .06, .08, .1. It means that we restricted to the couples (ith patent, jth class) such that p_ij ≥ p⁻.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Estimated values of θ^(tec) and θ^(sem) and corresponding standard errors obtained from a Maximum Likelihood estimator as presented in section Statistical Model.

https://doi.org/10.1371/journal.pone.0176310.t001

The choice of considering non-overlapping blocks (instead of overlapping blocks) is merely statistical. Ultimately, our interest is in the significance of the test over the whole period 1980–2007. Thus, we want to compute a global p-value. This can be done considering the local p-values (by local, we mean for instance computed on the period 2001–2005) assuming independence between them. This assumption is reasonable only if the blocks are non-overlapping. All of this can be found in S4 Text. Finally, note that from a statistical perspective, including overlapping blocks wouldn’t yield more information.

The values reported in Table 1 are overwhelmingly against the null hypothesis. The global estimates of θ^(sem) are significantly bigger than the estimate of θ^(tec) for all the considered thresholds. Although the corresponding p-values (which are also very close to 0) are not reported, it is also quite clear that the bigger the threshold, the higher the corresponding θ^(sem) is estimated. This is consistently seen for any period, and significant for the global period. This seems to indicate that when restricting to the couples (patent, class) with high semantic probability, the propension to cite patents from its own class θ^(sem) is increasing. We believe that this might provide extra information to patent officers when making their choice of citations. Indeed, they could look first to patents which belong to the same semantic class, especially when patents have high probability semantic values.

Note that the introduced model can be seen as a simple model of citations network growth conditional to a classification, which can be expressed as a stochastic block model (e.g. [45], [46]). The parameters are estimated computing the corresponding MLE. In view of [47], this can be thought as equivalent to maximizing modularity measures.