Validity of the Model

A key assumption for the validity of the model used in this study is that the core networks described do actually control variation in phenotype in natural systems. Intriguingly both the classical and the mutual inhibition type networks can be found as sub-networks of the Drosophila gap gene network [17]. The gap gene network of the Drosophila blastoderm has been shown to reduce variation in gene expression from Bicoid to the downstream GAP genes by the process of canalization [23], [24]. Indeed knock out of Kruppel, Knirps or Tailless increases this variation [25], [26]. Hence it can be demonstrated that for at least 2 core GRNs evaluated in this study there exists evidence that these networks control variation in gene expression and the resulting phenotype since the Gap genes control the segmentation of the embryo. The model is thus valid to explore evolutionary trajectories in developmental GRNs.

One criticism of the model we have used is that it does not utilize a typical fitness measure. Typical fitness measures involve some aspect of the phenotype that can be considered a selectable trait. For example the sharpness of the stripe of gene expression would be one such measure that could potentially give a selective advantage. These features have important influences on the likelihood that different routes are taken in genotype space. However the most important contribution to fitness is whether the genotype can achieve the function or not irrespective of how “well” it performs the function. Our average functional neutrality is based on the likelihood of a genotype that corresponds to that topology being functional. We have shown average functional neutrality of topologies correlates with the fraction of functional paths through them (Figure 3b). As such it is a valid statistical measure of the amount of RSE and restriction in evolutionary paths within a genotype-phenotype structure.

Furthermore this idea can be illustrated when we envisage our sampling in topology space as a density map of functional genotypes (topology with specific parameters) in underlying continuous genotype space (Figure 3e). Here the regions of space corresponding to the topologies with high average functional neutrality are dense, while those corresponding to the topologies with low average functional neutrality are sparse. For one to travel from a genotype in one dense region to a genotype in the other dense region directly through topologies 2 and 3, statistically many of the routes will involve a non functional intermediate as illustrated by the 2 RSE geometries in figure 3e (this point is confirmed by our mutational walks through parameter space; values on the edges in figure 3b). Hence though we have only scored this topological geometry as a single RSE, in the underlying continuous genotype space it may represent multiple RSEs. In the same way, though we have scored just a single extra dimensional bypass from topology 1 through 5, 6 and 7 to 4, in underlying continuous genotype space there maybe multiple routes as illustrated by the bold arrows in figure 3e. Taken together our measures of RSE and bypass frequency are likely to be underestimates of the actual amounts of RSE and bypasses in these genotype-phenotype maps. This further strengthens our conclusions that RSE and bypasses are frequent in genotype-phenotype maps for higher levels of biological function.

A General Theory of Permissive Mutations

Our analysis suggests one general theory for permissive mutations in gene regulatory networks and possibly other genotype-phenotype systems. There is an order by which mutations with opposing affect on functionality must occur. A mutation with a minor affect (the permissive mutation) must occur before a mutation with an opposite larger effect to maintain the system in the dynamic range as defined by the system configuration. For example in this work one aspect of the system configuration is the maximum and the minimum of the morphogen gradient which defines a dynamic range between 1 and 0.1. The minor permissive mutation shifts the functionality (stripe of gene expression) to the edge of the configuration limits (edge of the morphogen gradient or spatial boundary in our case). The probability of maintaining function with this mutation is greater than the probability of maintaining function with the large opposing mutation since the large mutation is likely to shift the functionality out of the dynamic range of the system (shift the stripe off the end of the spatial boundary so that the stripe only occurs between two morphogen thresholds both above 1– Figure 3c second panel). However once the permissive mutation has occurred, the scope of the dynamic range of the system is now much larger for the second larger mutation since its affect on functionality (the stripe) will push the system to the opposite side of the configuration limits. Hence the larger mutation can be a viable mutation, but only if the permissive mutation has already occurred. Note such situations would not arise if the morphogen gradient would not have configuration limits and could range from infinity to 0. However such boundaries or limits to the system like the concept of saturation for example are physically unavoidable aspects of GRN systems. As such the concept of smaller permissive mutations followed by larger mutations with opposing effect are probably a common explanation for extra-dimensional bypasses in both genotype-phenotype maps of GRNs and other genotype-phenotype systems.

Testable Hypothesis

How then can we experimentally test out hypothesis that mutually balanced interactions are responsible for RSE and interactions of opposing action yet differing strength are responsible for permissive mutations? Testing such hypothesis will require going beyond the identification of essential genes (for which data is currently available in multiple species) to a situation where essential individual gene-gene interactions are identified. A complementary experiment for validating our hypothesis would involve constructing our GRNs synthetically. If one could synthetically construct such networks then one could directly explore the affects of adding complementary positive and negative interactions of differing strength. We would predict for example that adding either alone (of similar strength) would diminish function, but in combination should have minimal effect. We would also predict that in general mutations of opposing effect but of different strength maintain function only if added in a particular order (minor permissive mutation first). At least 3 of our core networks have already been built in multiple species and others are currently being constructed [27]–[30]. The power of such constructions is that they only contain the basic core interactions needed for the function rather than other natural networks that have become baroque in nature (contain a more complicated architecture than seems necessary for the function [31]) probably through drift. Indeed our observation that extra dimensional bypasses are common in genotype-phenotype maps for GRNs suggests one explanation for why GRN networks have a baroque structure in the first place since there are many possibilities for adding functionally neutral gene-gene interactions which seemingly are not important for core function.

Replaying the Tape of Life at Higher Levels of Biological Organization?

How can the results of this study be compared to those results exploring similar phenomena in single proteins? A related phenomenon to permissive interactions with restricted evolutionary paths due to RSE has also been observed for specific examples in protein sequence-function relationships [14]. The existence of such a phenomenon in these diverse contexts suggests that permissive routes through higher genotypic dimensions are a general feature of the evolution of biological systems. However, one of the key findings of studies of evolutionary constraints in single proteins was that many intramolecular combinations are non-viable and therefore the trajectories open to evolution are limited and often the “tape of life” can be replayed such that that evolution often takes the same mutational path [13], [32], [33]. Here we have studied a higher level of biological organization and instead find that there are many more routes between functional genotypes (almost all incidences of 2 topologies linked via RSE have at least one bypass for example). This discrepancy between these conclusions could simply result from the fact that we have used a binary fitness function. Although many routes produce a functional stripe of gene expression, some routes may result in the production of a “better” stripe than others such that those routes are more likely. If a less abstract fitness function was used then involving some aspect of the quality of the stripe this may biased the likelihood of some specific paths over others. Alternatively the discrepancy may result from the fact that when considering a single protein the ways to increase fitness are greatly limited due to pleiotropy and conformational epistasis. Protein sequence space is discrete and any change can have a great effect on multiple aspects of the protein potentially destroying the function. Genotype space by contrast is continuous meaning that changes can be finer allowing for more oblique traverses without loss of function. The tape of life for the evolution of development at the level of GRNs then, may not be as predictable as that for protein function.

Enumerating All GRN Topologies

A topology can be represented in the form of a matrix w_ij where i and j represent the position in those matrices and values 1, −1 and 0 represent activation, repression and no interaction respectively. We generated all possible matrices that correspond to unlabelled topologies and then removed isometric equivalents by comparing them in all possible permutations. There are 19,683 gene network matrices before non-isometric topologies have been removed and this is reduced to 3,284 topologies in the fully enumerated set.

The morphogen gene is a gene that activates one of the genes of the GRN but is not affected by the GRN. Each GRN topology is represented multiple times with the morphogen feeding into the different genes (exact number depends on the amount of symmetry in the GRN topology). The morphogen is taken account of in the topology generation by extending the GRN matrix (i = i+1) to include the input from the morphogen (which is permutated independently). When the morphogen is included the number of isometric topologies increases from 3,284 to 9,710.

Creating an Atlas of GRNs by Including Explicit Neighbour Definitions

Two GRN topologies are considered neighbours in the atlas if the two GRN topologies are one Hamming distance apart (a single gene-gene interaction change). The Hamming distance can be measured by the following equation where(1)D is the Hamming distance between the matrices of two GRN topologies w and w′ whilst i and j represent the position in those matrices. The matrices are compared in every permutation and the lowest D of those permutations is taken as the Hamming distance. Hence two GRN topologies are neighbours if the gain or removal of any one interaction can transform one of the GRN topologies into the other.

The Gene Regulation Model

We employed a biologically-verified model of gene regulation for this problem, and therefore adapted the continuous mathematical model developed over the last 20 years by Reinitz et al [34] which quantitatively captures the spatio-temporal dynamics of gap gene patterning in response to the Bicoid morphogen gradient during Drosophila embryogenesis. The model is described by(2)where g_ij is the concentration of the ith gene in the jth cell, is a function defining the interaction amongst genes (which can take the form of a Michaelis-Menten, sigmoid or other non-linear input function), is a matrix containing the strength of gene-to-gene regulation parameters, M is the morphogen input described in more detail in the section “configuration of the spatial domain” below, is the Heaviside function (to prevent negative gene product production rates), is the diffusion constant for the ith gene which we use to represent local cell-cell signaling, is the decay rate (set to 0.05), and is a noise term which adds uniformly-distributed fluctuations (+/−1%) to the concentration of every gene in every cell at every time step. There is zero auto-correlation in the noise term. The parameters that could vary in the model were regulation , and diffusion . The input function describes the relationship between the activation and inhibition of a gene and its actual expression. The input function used in this work took the form of a Michaelis-Menten function which is defined by(3)where is the total input into the gene and is the output of the function.

The Discretized Form of the Equations

How the concentration x of a gene i will change in any given cell j at time t is described by(4)Here is the interaction matrix described earlier and is the input function. is the diffusion coefficient for each gene and is the decay parameter which is the same for every gene. The noise term selects a random number within a given range. is the Heaviside function where for and otherwise. Its purpose is to make sure the regulation term can only take positive values.

Parameter Range Distributions

For each GRN topology 30,000 different parameter sets were tested (a GRN topology with a specific set of parameters we called a genotype). There are up to 12 variable parameters for a 3 gene network; diffusion for each individual gene and then the strengths of the interaction values between the genes. The parameters are chosen randomly though biased towards lower numbers through a logarithmic probability distribution. The logarithmic probability distribution was implemented in order to take account of the fact that a small change in a small parameter value will have a greater effect on a network’s behaviour than a small change in a larger parameter value. The logarithmic probability distribution is described by(5)Where is a random number between 0 and 10,000 and is the parameter range and is the resulting parameter value. Parameter ranges are as follows; regulation 0–10 and diffusion 0–0.05.

Configuration of the Spatial Domain

The simulations take place on a theoretical one dimensional row of 32 cells. Zero-flux boundary conditions are used throughout this work. The simulation starts with every gene in every cell set to have a concentration of 0.1. This was necessary because the noise term used is a percentage noise term and thus if the concentration were always 0 at the start of the simulation then the products of any genes with positive feedbacks without any other input would remain at 0. The simulation is also initiated by the positive input from the morphogen gradient that does not change throughout the simulation.

The morphogen strength was chosen to give an approximate input range to the receiving gene of 10–50% of the maximal activation. The morphogen input is defined by(6)Where is the morphogen input, is the morphogen concentration in the left-most cell of the field, is the reduction of morphogen concentration in each subsequent cell of the morphogen gradient and is the cell position. For the 10–50% input range, and was used for the Michaelis-Menten function.

Stripe Forming Functional Definition

For a genotype (GRN topology with a specific parameter set) to be considered functional it had to reach an equilibrium (described in Supplementary Methods) and it had to produce a stripe of gene expression for at least one of the genes. For each gene we measured an abstraction of its gene expression over the one-dimensional field where each cell was defined as low or high. We defined a cell as low if the gene expression was below 10% of the maximum possible allowed by the model. We defined a cell as high if the gene expression was above 10% of the maximum gene expression allowed by the model. A gene was considered to have a stripe pattern if it had a single region of low for 2 consecutive cells followed by a single region of high for a maximum of 16 consecutive cells followed by a single region of low for at least 2 consecutive cells. The two low regions must occur at the extremities of the field. The definition is intentionally loose in the sense that the single stripe can be of any width up to 16 cells and be in any position in the spatial domain. This is because we are interested in the basic design principles of the system, not the details of how to control a specific width. Functional parameter sets that can produce the single stripe of gene expression we term “solutions”. Hence a single topology has multiple genotypes and can have multiple solutions. The number of solutions that each topology has is a measure of its mutational robustness. A GRN topology must have at least one solution to be considered functional.

Splitting the Functional Topologies into Subsets each Responsible for a Different Mechanism and Creating Average Functional Neutrality Landscapes

In order to explore whether changes in the mechanism by which a GRN is functioning is responsible for the prevalence of RSE in an average functional neutrality landscape, we split the functional GRN topologies into 6 categories, each corresponding to the 6 different mechanisms we identified in our previous study [17]. If a GRN topology could perform a mechanism for at least one parameter set then it was included in that particular subset. Therefore a GRN topology can be present in multiple subsets. GRN topologies were assigned to mechanistic classes using the method for mapping mechanisms to the complexity atlas described in Supplementary Methods. Each subset is then used to build an average functional neutrality landscape for each particular mechanism. The total number of functional GRN topologies was 471. The number of GRN topologies in each of the average functional neutrality landscapes was as follows; Incoherent Feed-Forward type 1 (97), Mutual Inhibition (44), Frozen Oscillator (27), Overlapping Domains (56), Bi-stable (109) and Classical (39). Topologies are assigned an average functional neutrality score based on their parameter robustness (Number of parameter sets that successfully produced the stripe of gene expression). Topologies are vertices in the landscapes and they are connected by edges based on the neighbor definitions of the atlas described earlier.

Calculating the Extent of RSE

In order to calculate the extent of RSE in the individual average functional neutrality landscapes we analyzed all pair-wise combinations of topologies in the average functional neutrality landscape to see if they conform to the V-shaped viability/fitness geometry. The V-shaped viability/fitness geometry of RSE is defined as two GRN topologies (topologies A and B) that are two hamming distances apart that have exactly two direct intermediate topologies. These intermediate topologies both must have significantly lower average functional neutrality than topologies A and B. Significantly low average functional neutrality is defined by(7)Where is the average functional neutrality of topology A (which is defined as the number of functional parameter sets for that topology) and is the standard deviation of the average functional neutrality of topology A, assuming a binomial distribution. The standard deviation of a binomial distribution is described by(8)Where is the number of trials and p is the probability of functionality (the true average functional neutrality). Since we do not know the true average functional neutrality, this measure is bootstrapped using the measured average functional neutrality as a proportion of the total number of parameter sets tested (number of trials, ) giving(9)

Calculating the Extent of Permissive Mutations/extra-dimensional Bypasses

In order to calculate the extent of extra-dimensional bypasses between pairs of GRN topologies that have the RSE V-shaped viability/fitness geometry we analyzed all mutational routes between these pairs of topologies. For a route to be considered an extra-dimensional bypass every GRN topology within the route must have average functional neutrality within one standard deviation of topologies A and B as described in equation 7. For each pair-wise combination of GRN topology showing the RSE geometry we record the length of each extra-dimensional bypass (measured by the number of gene-gene interaction changes). The direct permissive mutations are those with a bypass length of 4 since they involve the 2 original mutations required to change topology A into B (and vice versa) and the two other mutations that involve the addition and removal of the permissive mutation.

Performing Mutational Walks between Topologies

We measured the fraction of mutations from a functional parameter set of one topology to a functional parameter set of another topology (specifically in the direction of the arrows in figure 3b). We start with a functional parameter set of our reference topology and remove or add the appropriate interaction. If we add an interaction, we give the new interaction the appropriate sign and with a random value using the criteria that we describe in parameter range distributions above. If we remove an interaction the value is simply set to 0. We then re-simulate this mutated genotype and ask if it is functional using our functional stripe definition described above. We do this 100 times for every specific parameter set and calculate the proportion of tests that are functional. For example we perform 74,400 tests of mutating topology 1 to topology 5. The proportions of functional tests are found in brackets on the edges of figure 3b.