Simulations of multi-layered network models evolving towards input-output goals

We begin with a simple linear model of a multi-layered network and later extend this framework to nonlinear models as well. The network in our model is a system that receives an input vector , performs L consecutive stages of processing; each stage produces an intermediate , ,… where the product of the final processing layer is the network output vector . Here, we assume each of these processing layers can be described by a linear transformation A₁, A₂,…A_L, such that the system output is . Each matrix contains all possible interaction intensities between network nodes at two consecutive layers. For example the matrix entry A^(l)_ij represents the effect of the j-th node in layer l on the i-th node in layer l + 1. In this model, connections are only possible between a node and any node at the next layer. Connections within layers or backward connections are not allowed. For illustration see Fig. 1.

As a concrete biological example, one may think of a metabolic network. The input vector is the number of nutrient molecules of different types that are consumed by the organism. Taking the example of carbohydrate metabolism: elements of the input vector represent the amounts of the various sugars consumed: s₁ could be number of glucose molecules consumed, s₂ number of lactose, etc. Sugar metabolism applies a series of enzymatic reactions which first break complex sugars into simpler ones (monosaccharides) and then converts them to either of several possible output products: ATP (energy source for short term usage), glycogen (carbohydrate storage) or other sugars, for example 5-carbon sugars (pentose) used for the synthesis of nucleotides, nucleic acids and aromatic amino acids. Intermediate nodes in the model represent intermediate sugar metabolism products, such as glucose 6-phosphate, fructose 6-phosphate, pyruvate etc. [10]. The output vector represents the numbers of molecules produced using the consumed carbohydrates: v₁ could be number of ATP molecules, v₂ number of glycogen molecules, v₃ number of ribose-5-phosphate, etc. This is admittedly a simplified description of sugar metabolism. For example, it does not take into account the hierarchical order of different sugars uptake, or metabolic cycles. Yet, it captures the degeneracy which enables replacing one sugar by another and still obtaining the very same output products. Related models have been useful for understanding multi-layered biological networks in diverse contexts, such as metabolic, gene regulatory and signal transduction networks [33,38,46,48–57].

In the linear model, the total input-output relationship of the network is given by the product of the matrices A₁, A₂,… A_L, which represents the overall transfer of signals from the first (input) to the last (output) layer. Employing this formalism, we evolve these networks to match a desired goal—given by a matrix G. The goal matrix describes the desired output for any possible vector of inputs, and thus defines the entire input-output function. The dimension of the goal matrix G corresponds to the number of system inputs and outputs D_output × D_input. We also extended this model to describe a gradually growing network for which the goal dimensions also change (see S1 Text).

The fitness of a network equals the distance between the total effect of the network (the product of the matrices) and the desired goal G, namely F = − || A^(L) A^(L−1) … A⁽¹⁾ – G ||. Note that each goal can be satisfied by an infinite number of matrix combinations that are all equally fit. For example, if G is the unit matrix G = I and L = 2, all pairs of matrices that are inverse to each other A⁽²⁾ = [A⁽¹⁾]⁻¹ satisfy the goal, because A⁽²⁾ × A⁽¹⁾ = I.

To evolve the networks, we use a standard evolutionary simulation framework [37,58–60]. Briefly, the simulation starts with a population of N networks, each described by a set of L matrices. At each generation the networks are duplicated and mutated with some probability resulting in modified interaction intensities. A mutation means a change to an element of one of the matrices. Fitness is then evaluated for each structure in comparison to a goal. N individuals are then selected to form the next generation of the population, such that fitter individuals are more likely to be selected. This process is repeated, until high fitness evolves (see Methods section for more details).

Guided by studies on the evolution of modularity, we tested evolutionary situations which are biased to reduce or eliminate interactions. Such a mechanism is a product-rule mutation scheme in which elements of the matrices are multiplied by a random number drawn from a normal distribution (as opposed to a sum-rule mutation where a random number is added instead of multiplied) [29]. Product-rule mutations are a more realistic description of the way that DNA mutations affect biochemical parameters than sum-rule mutations [41–47]. Biological mutations are more likely to decrease existing interactions than to create novel ones that did not exist before [61–63]. This property is captured by product-rule mutations (but not by sum-rule) [29]. With this mechanism, evolution finds networks satisfying the goal which are highly sparse—that is, networks with a small number of significant interactions [29,46]. As controls, we also simulated evolution with sum-rule mutations (in which a random number is added to matrix elements).

Bow-tie architectures evolve when the goal is rank deficient

In the example of carbohydrate metabolism, different input sugars (glucose, lactose etc.) can be used to produce any of the final products (ATP, glycogen, ribose-6P). If one only examines the output molecules, one cannot tell which original sugar was their source. This degeneracy can be mathematically represented as a goal matrix with linearly dependent rows. To study the effect of these dependencies on the network structures that can evolve, we tested evolution towards goals described by matrices with different ranks. The rank r is the number of linearly independent rows in the matrix. The rank of the goal matrix is full, if all rows of the matrix are independent. If some of the rows are dependent, the matrix has deficient rank—a rank smaller than the full rank. Deficient rank means that the input-output transformation maps inputs to a limited subspace of outputs, of dimension r. Below, we discuss the implications of this concept also for nonlinear systems. As an example of a 3 × 3 matrix with rank r = 1 consider the following matrix whose two last rows are given by a constant multiplying the first row:

Rank-deficient matrices can be decomposed into a product of (generally non-square) matrices , whose smallest dimension equals the rank of the goal, namely min_i(D_i) = r. Because the rank of the goal matrix is smaller than its dimension r < D, this decomposition is equivalent to a narrow waist, whose width is equal to the goal matrix rank r. As a simple example consider the 3 × 3 goal above. It is decomposable into a product of a column vector by a row vector:

This decomposition represents a 3-layer network, whose intermediate layer has only one active node—namely has a bow-tie structure (see left scheme in Fig. 1B). Importantly, however rank-deficient matrices can also be decomposed into products of full matrices. For any choice of A⁽¹⁾, one can find A⁽²⁾ = [A⁽¹⁾]⁻¹ G, as long as A⁽¹⁾ is invertible. In fact most random choices of A⁽¹⁾ will yield full-matrix decompositions which represent a non-bow-tie architecture (right scheme in Fig. 1B).

More generally, let be a goal matrix with rank r = rank(G). Let there be a decomposition of into a product of L matrices G = A^(L)A^(L−1)…A⁽¹⁾. This L matrix decomposition is a representation of a network that has L + 1 layers of nodes, whereas each matrix represents the interaction intensities between all nodes in two adjacent layers. If and only if G has deficient rank, it can be decomposed into a product of matrices having dimensions smaller than the goal dimensions. This means that the matrices represent a network with intermediate layers that consist of fewer active nodes than the number of inputs and outputs to the network. Otherwise, if the matrix has full rank, each layer must have a number of active nodes which is at least as large as the rank, making a bow-tie impossible in a case of full rank. This argument follows from the fact that matrix multiplication cannot increase rank, i.e. rank(A B) ≤ min (rank (A), rank (B)) for any two matrices A and B [64].

The narrowest layer in the network is termed the waist [3]. The waist can be narrow because of the low rank that allows compressing the inputs down to fewer nodes, and then computing the outputs based on those nodes. While in principle such parsimonious bow-tie decompositions exist for every rank-deficient matrix, they constitute only a small fraction of all possible decompositions. Thus, the question remains whether evolutionary dynamics can find the solution with a bow-tie out of the infinite number of non-bow-tie solutions.

To address this question, we evolved networks towards rank-deficient goals, with and without the product-rule mutation scheme described above. We studied goals of dimension D = 6–8 consisting of L = 4–6 matrices, tested 4–8 different goals for each dimension, and evolved networks towards each goal in 100–3000 repeated simulations, each starting from different random initial conditions. We found that a rank-deficient goal together with product-rule mutations gave rise to networks that satisfy the goal and show bow-tie architectures. Full rank goals always led to evolved networks that satisfied the goal but had no bow-tie architecture at all, namely all layers had exactly the same number of nodes as the input and output layers. Rank-deficient goals with a different mutational scheme (sum rule mutations) could sometimes lead to architectures in which intermediate layers had fewer nodes than the input and output layers, but these were mostly not as narrow as the goal rank. When noise was introduced to the simulations (see below), the sum-mutation scheme led to full (non bow-tie) structures even under low noise levels. We conclude that a bow-tie evolves when (i) the goal has deficient rank, and (ii) mutation rule is product and not sum. Bow-tie width equals the rank of the goal.

For example, consider a network with 5 layers of nodes (L = 4 matrices of interaction layers), each consisting of 6 nodes (D_input = D_output = D = 6). We simulated their evolution towards goals of different ranks between 1 and full rank (r = 1–6). We repeated the simulation 700 times for each goal starting from different random matrix initial conditions. We then analyzed the number of active nodes at each layer. Since in a numerical simulation we do not obtain exact zeroes, but rather very small values, we defined active nodes as nodes who, if removed (incoming and outgoing interactions of the node set to zero) have a larger than 0.1% relative effect on fitness (see Methods).

We find that the number of active nodes is smallest on average at the middle layer. The number of active nodes at this waist is most often equal to the rank of the goal (Fig. 2), and never lower than this rank. The first and last layers are constrained to have exactly D active nodes by the definition of the problem (Fig. 3). In a minority of cases (~20%, see Table 1 in S1 Text) the waist showed more active nodes than the rank of the goal. For comparison, if the mutational mechanism is not biased to decrease interactions (i.e. sum-mutations) the vast majority (94%-97%) of the runs ended with mid-layer which had more elements than the rank of the goal (Table 1 in S1 Text). We show a representative example of a network configuration obtained in simulation in Fig. 1C.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Product-rule mutations and goal which is not full rank can lead to bow-tie architecture.

We show simulation results of networks with L = 4 (5 layers of nodes) and 6 nodes in each layer (D = 6). We performed 4 different sets of repeated simulations with goals of different ranks = 1,2,3 or 6. We illustrate the histograms of layer width for each set of runs. Each column in this figure shows simulation results for a different goal, and each row shows a different network layer. The number of active nodes in middle layers varies depending on the goal. The minimal number of nodes in intermediate layers (“waist”) is bounded from below by the rank of the goal. The waist width could be higher than the rank, because not all runs reach the most minimal configuration, but it cannot be lower. For example, it can be as low as 1 if the goal rank equals 1 (left column), but it is always 6 if the goal is full rank, demonstrating that no bow-tie can evolve with a full-rank goal. Simulation parameters: 3000 repeats for rank 1 and 2, 1500 repeats for rank 3 and 700 repeats for rank 6. Only runs that reached a fitness value less than 0.01 from the optimum were considered in the analysis. Product mutations were drawn from a Gaussian distribution with σ = 0.1, element-wise mutation rate p = 0.05 / D², tournament selection with s = 4.

https://doi.org/10.1371/journal.pcbi.1004055.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. The waist is most likely to evolve in the middle layer (for equal number of inputs and outputs).

Top: Median number of nodes at each layer. Different curves represent results for goals of different ranks. Due to symmetry considerations, the waist is most likely to evolve in the middle layer of nodes. Results refer to the same simulations as in the previous figure. Estimation of error in median calculation by bootstrapping resulted in negligible error. Bottom: examples of possible network structures evolved with goals having different ranks 1,2,3 and 6, illustrating how the width of waist depends on the goal rank.

https://doi.org/10.1371/journal.pcbi.1004055.g003

We tested the sensitivity of this mechanism for bow-tie evolution to model parameters. A bow-tie was obtained under a wide range of values of selection intensity, mutation size, mutation rate and population size that spanned 1.3 decades (mutation rate) to 2 decades (mutation size) (Fig. 4; see S1 Text for more details). We also tested the sensitivity of the structure obtained to the evolutionary goal by comparing simulation results with different goals having the same rank. We find that the location and width of the waist are insensitive to the choice of the goal (see Fig. 9 in S1 Text).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. bow-tie architecture is obtained under a broad range of evolutionary parameters.

A We tested the existence and width of bow-ties under a broad range of parameter values. We illustrate here the mean and standard deviation of the bow-tie width for various values of mutation rate, mutation size, population size and selection intensity. Bow-ties were obtained in all cases. The width of the bow-tie showed little sensitivity to the parameter values. Each point is based on 50 independent repeats of the simulation. Parameter values tested: population size = [50, 100, 250, 500]; mutation size = [0.01, 0.05, 0.1, 0.2, 0.5, 1]; mutation rate = [1, 0.25, 0.1, 0.05]/LD², tournament size s = [2, 4, 6, 8].

https://doi.org/10.1371/journal.pcbi.1004055.g004

We tested the location of the waist in simulations of multi-layered networks with equal number of inputs and outputs (L = 4, D = 6; L = 6, D = 8). While in principle the waist could reside at any layer between the input and output layers, in practice, it falls most often in the middle layer. Intuitively, this can be explained by symmetry considerations: The mutational mechanism works uniformly on all layers to eliminate connections. While the dimensions of the goal matrix constrain the number of active nodes at the network boundary layers (input and output), connections near the middle layer are least “protected” and thus mostly prone to removal, resulting in the network waist being on average in the middle layer.

Bow-tie architecture also evolves under temporal noise

Biological networks are often prone to fluctuations, for example due to temporal variations in internal molecule numbers or environmental fluctuations. We thus tested the robustness of the suggested evolutionary mechanism to fluctuations in either the goal or the interaction intensities. We started by testing the sensitivity of the evolutionary mechanism to rank accuracy by perturbing the rank-deficient goal matrix, yet keeping the goal constant throughout every simulation run. This produced goal matrices that are ‘almost rank deficient’: full rank, but with some of the eigenvalues close to zero. The noise strength is given by the difference between the norms of the noisy and clean goals divided by the norm of the clean goal (see Methods). We find that for noise strength up to about 1%, bow-tie architecture with middle layers whose width equals the goal rank were reached in most simulation runs, just as in the absence of noise. Thus, our evolutionary simulation is robust to small perturbations to exactly rank-deficient goals—see Fig. 5 for illustration and compare to Fig. 2 with no noise. The median waist size increased above the clean rank when noise intensity increased above 1% (see Figs. 14–15 in S1 Text for the dependence of bow-tie on the noise level). For estimation of the noise magnitude in biological networks, see S1 Text.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Bow-tie evolves even if the goal is only approximately of deficient rank.

We show simulation results when the goal consisted of a matrix of deficient rank (1, 2 or 3) to which some level of noise was added (see Methods), so mathematically speaking goals had full rank, such that some of the eigenvalues were relatively small. Remarkably, here too a bow-tie architecture evolved, however the width of the waist was not as narrow as if the goal had exact noiseless deficient rank (compare to Fig. 2). For each goal rank we calculated layer activity statistics based on 1500 different runs (each having a different goal, but with same noise statistics). Noise level here was 1% (averaged over all runs analyzed) for all ranks. This result demonstrates that the evolutionary process can expose a deficient goal rank even when noise is added, as is expected to be the case in realistic systems. Other parameters are the same as in Fig. 2.

https://doi.org/10.1371/journal.pcbi.1004055.g005

While in the previous scenario the goal rank was noisy, but remained constant throughout every simulation run, temporal fluctuations are also ubiquitous in biological networks. To test the effect of temporal fluctuations, we added statistically independent noise realizations (white noise) to all matrix entries (and also to the goal) at each generation. The fitness evaluation then reads: F = − || (A^(L) + ε_L)(A^(L−1) + ε_L−1)…(A⁽¹⁾ + ε₁) – (G + ε_G) ||, where ε_i are independent noise realizations. Since this noise changes at a higher frequency than the typical evolutionary timescale (the mutation rate), we expected that the system will be able to filter it out to some extent. Since here the noise affects all network components and not only the goal, we refer to the induced fluctuations in fitness as a global measure of the noise intensity. We compared the ability of evolution with either product or sum mutations to cope with this temporal noise. We find that product-mutations filter out the noise much more efficiently than sum-mutations. When the clean goal rank was 1, the network structure evolved by product-rule evolutionary scheme was unaffected until the relative magnitude of induced fluctuations in fitness reached values of 0.3 (std/mean). Sum mutations, in contrast, led to bow-tie of width 3 (compared to the minimal width in this case, 1) even in the absence of noise; bow-tie width sharply increased to 5 when temporal noise was added. Since complete absence of noise is a non-realistic scenario in biological systems, we conclude that sum-mutations cannot account for bow-tie evolution. See Fig. 16 in S1 Text for illustration.

Bow-ties can evolve in nonlinear information transmission models

Finally, we asked whether the present mechanism would apply in a nonlinear network model. While goal rank is a straightforward measure of dimensionality in linear systems, the concept of rank is more elusive when it comes to nonlinear systems. Yet, one can intuitively think that a similar concept could exist there too. To test this hypothesis we employed a well-studied problem of image analysis using perceptron nonlinear neural networks [65,66]. In this problem, each node integrates over weighted inputs and produces an output which is passed through a non-linear transfer function, u^(l+1) = f (A^(l)u^(l) – T^(l+1)), where A^(l) and T^(l) are the weight matrix and corresponding set of thresholds in the l-th layer, and u^(l) is the set of inputs propagated from the previous layer (see Methods).

We evolved the networks towards a goal of identifying features in a 2 × 2 retina with Boolean pixel values (D₂ = 4 inputs) (Fig. 6). Low dimensionality was achieved by defining as a goal four outputs that depend only on two features of the image. The four required Boolean outputs were: (a) at least one pixel in the left retina column, (b) at least one pixel in the right column, (c) pixels in both left and right columns, (d) pixel(s) in the left or in the right columns. These four outputs can be fully represented by only two features: (a) and (b), making the 4-dimensional input space redundant. Thus, the effective “rank” here is r = 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Bow-tie can evolve for a nonlinear input-output relation too, if the input can be more compactly represented with no effect on the output.

We show simulation results of a simple nonlinear problem mimicking a 4-pixel retina. (A) Problem definition: The retina has four inputs (one for each pixel, that can be either black or white), four outputs and two internal processing layers. The retina is evolved so that its outputs detect whether there is (i) an object on the left side (at least one pixel in the left column is black), (ii) on the right side (at least one pixel in the right column is black), (iii) left AND right objects, (iv) left OR right objects, correspondingly. Inset: in contrast to previous problems, here each node performs a nonlinear transformation of the sum of weighted inputs: u^(l+1) = f (A^(l) u^(l) – T^(l+1)), where A^(l) and T^(l) are the weight matrix and set of thresholds in the l-th layer. (B) Typical example of simulation results. Apparently, two bits of information are sufficient to fully describe the four required outputs in this model. Indeed, the network evolved so that it has only two active nodes in the second layer (red circles).

https://doi.org/10.1371/journal.pcbi.1004055.g006

Simulations evolving the weights {A_ij^(l)} and thresholds {T_i^(l)} values using product-rule mutations that led to nearly perfect solutions (fitness of less than 10⁻⁴ from the optimum) mostly had a narrow waist: one of the intermediate layers had only two active nodes in 75% of the runs (see Table 1 in S1 Text). For comparison, simulations with a mutation rule that was not biased to eliminate interactions (sum-rule) were much less likely to lead to networks with a narrow waist (this was observed in only 45% of runs). Detailed statistics over 500 runs of network structures obtained with either mutational scheme is presented in Fig. 12 in S1 Text.

Evolutionary simulation

The evolutionary simulation was written in Matlab using standard framework [58–60]. The source codes and analysis scripts are available as supporting materials. We initialized the population of matrices by drawing their N ⋅ LD ² terms from a uniform distribution. Population size was set to N = 100. Each “individual” consists of a set of L matrices. In each generation the population was duplicated. One of the copies was kept intact, and elements of the other copy had a probability p to be mutated—as we explain below. Fitness of each of the 2N individuals was evaluated by F = − || A^(L)A^{(L – 1)} … A⁽¹⁾ – G ||, where || ⋅ || denotes the sum of squares of elements [82]. The best possible fitness is zero, achieved if A^(L) A^{(L − 1)} …A⁽¹⁾ = G exactly. Otherwise, fitness values are negative. We constructed the goal matrices from combinations of ‘0’ and ‘10’ terms. We tested goals of different ranks and different internal structures and found no sensitivity for goal details other than its rank (see S1 Text). N individuals are selected out of the 2N population of original and mutated ones, based on their fitness (see below). This mutation–selection process was repeated until the simulation stopping condition was satisfied (either a preset number of generations or when mean population fitness was within 0.01 of the optimum).

Data analysis

Repeated simulations were run using the same parameters, where at every single run the Matlab random seed was initialized to a different value. Consequently, each run starts from different initial conditions and uses different mutational realizations. In the analysis, we checked whether the runs converged. Only runs that gave results within 0.01 from the optimum were considered in the analysis. We then analyzed in each run the number of active nodes in the layer (see below). In the figures we show either the median number or histogram of active nodes per layer, when applicable.

Retina problem

We tested the evolution of bow-tie networks in this non-linear problem which resembles standard neural network studies [39,65,84]. We defined a problem with 4 inputs and 4 outputs and 2 internal processing layers consisting of 4 nodes each. The inputs represent a 4-pixel retina, where each pixel could be either black or white, as described in the results section.

The evolutionary simulation followed a similar procedure to the linear problem described above. Mutation, selection, and data analysis methods were similar to the ones used in the linear problem as described above. The main difference is that the output of each layer was not a linear function of the inputs as before, but rather a non-linear function u^{(l + 1)} = f (A^(l) u^(l) – T^{(l + 1)}), where A^(l) and T^(l) are the weights matrix and corresponding set of thresholds in the l-th layer correspondingly, and u^(l) is the set of inputs propagated from the previous layer. The non-linear transfer function f was rescaled to range between 0 and 1, f(x) = (1 + tanh(x)) / 2. The result of this computation is fed to the next layer until the last (output) layer is reached. In non-linear systems the evolutionary goal cannot be described by a single goal matrix as in the linear case. Rather, it is defined by pairs of input /output relations. The evolutionary simulation tested all possible inputs simultaneously, and evolved the network parameters to provide the correct output in each case. The fitness was defined as the difference between the network output and the desired output, in similarity to the linear model and then averaged over all possible input/output pairs. Inputs and outputs were encoded by Boolean vectors. Internal layer calculation used continuous values, but simulations could reach very high precision (≤ 10 ⁻¹⁰ from the optimum). Simulations were run for 10⁴ generations. Only runs that reached fitness within 10⁻⁴ of the optimum were considered in the analysis.

The retina simulation was written in Wolfram Mathematica. We initialized the population of matrices and corresponding thresholds by drawing their N ⋅ LD(D + 1) terms from a uniform distribution in the range [-2,+2]. Population size was set as N = 100. In each generation the population was duplicated. One of the copies was kept intact, and elements of the other copy had a per-term probability p = 0.2 to be mutated. The mutation was implemented through multiplying the mutated term by a random number drawn from a normal distribution with mean 1 and std 0.5 (thus a probability of about q = 0.02 to change sign).

To determine active nodes in this case, we begin by setting each weight to zero in its turn A^(l) _ij = 0 leaving all other terms intact. This procedure was not applied to the threshold values T_i^(l) because a node may be left in the network, even if no inputs are propagated through it from an upper layer. In these cases the role of such a node is to introduce a constant bias set by its threshold. We then calculate the fitness value of the modified network and define the difference compared to the original fitness value: . We compare ΔF / F between all weights located at the same layer. A network interaction whose relative effect on fitness is less than 10⁻⁴ was set to zero. A node whose entire set of outgoing weights was set to zero was considered inactive.