https://orcid.org/0000-0002-8049-6186
Figures
Abstract
Spatial biological networks are abundant on all scales of life, from single cells to ecosystems, and perform various important functions including signal transmission and nutrient transport. These biological functions depend on the architecture of the network, which emerges as the result of a dynamic, feedback-driven developmental process. While cell behavior during growth can be genetically encoded, the resulting network structure depends on spatial constraints and tissue architecture. Since network growth is often difficult to observe experimentally, computer simulations can help to understand how local cell behavior determines the resulting network architecture. We present here a computational framework based on directional statistics to model network formation in space and time under arbitrary spatial constraints. Growth is described as a biased correlated random walk where direction and branching depend on the local environmental conditions and constraints, which are presented as 3D multilayer grid. To demonstrate the application of our tool, we perform growth simulations of a dense network between cells and compare the results to experimental data from osteocyte networks in bone. Our generic framework might help to better understand how network patterns depend on spatial constraints, or to identify the biological cause of deviations from healthy network function.
Author summary
We present a novel modeling approach and computational implementation to better understand the development of spatial biological networks under the influence of external signals. Our tool allows us to study the relationship between local biological growth parameters and the emerging macroscopic network function using simulations. This computational approach can generate plausible network graphs that take local feedback into account and provide a basis for comparative studies using graph-based methods.
Citation: Paul TJ, Kollmannsberger P (2020) Biological network growth in complex environments: A computational framework. PLoS Comput Biol 16(11): e1008003. https://doi.org/10.1371/journal.pcbi.1008003
Editor: Pedro Mendes, University of Connecticut School of Medicine, UNITED STATES
Received: May 26, 2020; Accepted: October 29, 2020; Published: November 30, 2020
Copyright: © 2020 Paul, Kollmannsberger. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All code and data to reproduce the results are available from https://github.com/CIA-CCTB/pythrahyper_net.
Funding: This publication was supported by the Open Access Publication Fund of the University of Wuerzburg.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Complex biological networks such as the neural connectome are striking examples of large-scale functional structures arising from a locally controlled growth process [1, 2]. The resulting network architecture is not only genetically determined, but depends on biological and physical interactions with the microenvironment during the growth process [3, 4]. In this context, evolution has shaped diverse spatial networks on all length scales, from the cytoskeletal network in cells [5], to multicellular networks such as the vascular system [6] or the osteocyte lacuno-canalicular network [7], to macroscopic networks of slime molds [8], mycelia [9] and plants [10]. Sophisticated imaging techniques together with large-scale automated analysis provide increasingly detailed views of the architecture of biological networks, revealing e.g. how neurons are wired together in the brain [11]. After extracting the topological connectivity from such image data, quantitative methods from the physics of complex networks can be applied to compare different types of networks and to uncover common organizational principles [12–15]. To understand the functional role of networks in the context of evolution, however, it is not sufficient to characterize static network structure [1]: we need to be able to infer the dynamics of the underlying biological growth process that defines this structure.
This poses a major challenge, since the local dynamics of the growth process is in most cases not experimentally accessible, except for simple model systems. Here, computer simulations provide a solution: they allow to connect the observed patterns to the underlying process by performing in-silico experiments. Different hypotheses can be tested by systematically varying the local growth rules in the simulation and analyzing the resulting network patterns. Such a computational approach also helps to overcome an important limitation of network physics with respect to spatial networks: the properties of complex networks are often calculated with respect to canonical random graphs such as the Erdös-Renyi [16] or Watts-Strogatz model [17]. These models are defined by the topological structure in an adjacency matrix, but usually neglect spatial constraints [18, 19]. A generic model of network growth under spatial constraints would thus address two important problems: enabling a meaningful quantification of spatial network patterns, and linking the observed patterns to the underlying biological growth process.
The first computer simulation of spatial network development, published in 1967 [20], already included the role of active and repulsive cues. Since then, several more advanced approaches to model spatial network growth have been developed. One type of model implements network growth by formation of edges between pre-existing nodes in space [21, 22] and was e.g. used to develop large-scale models of branching neurons [23] or leaf vascular networks [24]. Another type of approach is based on growth processes emanating from predefined seed points with branching and merging and has been used to model neural network development [25, 26], fibrous materials [27] or the development of branching organ structures [28]. Most of these existing models are targeted towards a specific domain such as neural networks and synapse formation, or include only simplified interactions with the environment. The current state of modeling spatial network growth was recently summarized in [1], arguing that there is “a crucial lack of theoretical models”.
The aim of this work is to develop a generic framework for biological network development in space and time under arbitrary spatial constraints. We propose a probabilistic agent-based model to describe individual growth processes as biased, correlated random motion with rules for branching, bifurcation, merging and termination. We apply mathematical concepts from directional statistics to describe the influence of external cues on the direction of individual growth processes without restricting the model by making too specific assumptions about these cues. Structural and geometric factors are obtained directly from real image data rather than formulated explicitly. The model is implemented as a computational framework that allows us to perform simulations for diverse types of biological networks on all scales, and to monitor the evolving spatial structure and connectivity.
Results
Growth process
The elementary process of biological network formation is the outgrowth of new edges in space from existing nodes, such as cells. Each individual growth agent, for example a neuronal growth cone, randomly explores space and senses soluble signals as well as physical cues and obstacles. This process can be formulated as a biased correlated random walk [29]: the movement of edges is biased by attractive or repulsive signals and structural cues, and correlated due to directional persistence of edge growth. Edges turn into trees if branching and/or bifurcations occur, such as in growing neurons, osteocytes or sprouting angiogenesis. Finally, if edges can join other edges and form new nodes (e.g. synapses), a connected network emerges. The elements of this discretized, agent-based formalization of spatial network development are summarized in Fig 1. In the following, we describe our mathematical approach for modeling the growth process.
Two growth agents (green and blue, e.g. growth cones, filopodia, cells) start at time ti and randomly explore their environment in a biased correlated random walk. The environment is represented as a discrete grid (grey lines), while the network grows in continuous space. Blue and green dots correspond to subsequent positions of the growth cones. Cones can branch off new daughter cones, bifurcate/split in two new cones, merge with existing edges, or terminate. These events leads to formation of new nodes in the network, shown as red dots. Over time, the individual processes form a connected network of nodes and edges in space.
In general, such a stochastic motion under the influence of external forces is described by a partial differential equation (PDE), the Fokker-Planck equation [30] corresponding to the Langevin equation [31]:
(1)
Eq 1 is a drift-diffusion equation of motion where the k-dimensional vector is a drift, and D the anisotropic and diagonal, k × k- dimensional diffusion tensor. This equation describes the time development of a probability density function
of diffusing particles under the influence of a drift
and diffusion
. If drift and diffusion are both constant in time and space, and if the initial condition
is the k-dimensional Dirac delta function, then Eq 1 solves to
(2)
Eq 2 describes the time development of a random movement for constant drift and isotropic diffusion starting at initial position . In a complex environment, however, both drift and diffusion can depend on time and position, and there is no simple analytic solution for the time development. In this case, a numerical approach can be taken by sampling individual movement traces as non-Markovian correlated processes [32]. The drift-diffusion movement is then discretized into a stepwise process where for each growth step, a new direction is drawn from a probability density function (PDF). For constant drift and diffusion (Eq 2), the corresponding PDF, or growth kernel, is a Multivariate Gaussian distribution (MVG) with mean vector
, covariance Σ = D t and determinant
. In the general case, assuming that the spatio-temporal change for
and
is small between two consecutive time steps, every discrete growth step has a different time- and position dependent PDF. The fundamental solutions Eq 2 of the PDE (Eq 1), or Eq 3 for non-diagonal covariance, can thus be used as random kernels in each step from which the subsequent growth directions are drawn. Their
and Σ can be expanded so that they encode for all spatial signals and constraints. A more detailed derivation of this approach starting from Eq 1 is provided as supporting information in S1 Text.
Substrate and signaling cues
The individual PDFs that describe all the different internal and external constraints and cues that act on an edge at position are formulated using the Multivariate Gaussian Distribution (MGD) as kernel:
(3)
The MGD is a probability density function from which random vectors of dimension k are drawn. Such a vector is shifted by the mean vector
and anisotropically distributed. The anisotropic distribution of directions is described by the metric of the k × k covariance matrix Σ with determinant |Σ| and for its eigendecomposition as real symmetric matrix Σdiag = A−T ΣA with eigenvectors
and eigenvalues Σdiag = diag(λ1 < ⋯ < λk).
Depending on the edge’s current position , the mean vector
points to the most likely direction the edge will be moved in the subsequent step, while the covariance Σ reflects the impact of the local structure onto its movement in the complex environment.
Following the mathematical rules for distributions, the PDF of a random vector that is the sum of independent random vectors is the convolution of the PDFs of the individual random vectors:
(4)
All local, external signals and structural properties that are described by individual PDFs in the form of a MGD can thus be combined into a single PDF that still has the form of a MGD [33]. As a result, a new growth direction that includes the contribution of all cues and constrains can be drawn from this combined MGD with mean vector
(5)
and covariance
(6)
These parameter pairs can represent different external cues, the geometric constraints of the substrate, as well as edge-specific properties. In Table 1, we give some examples for external and internal cues and the structure of the environment that can be represented by such “Mean Vector—Covariance” pairs.
Typical examples of external cues are drift vector fields and ensembles of signaling molecules. For the case of a constant external vector field, the mean vector points towards the local field orientation, while its length scales with the field’s strength. Its corresponding covariance is symmetric, proportional to the unit matrix , and the entries tend to zero (σ1,…,i → 0). The MGD is then a k-dimensional delta distribution that biases the growing edge in the direction of the vector field.
Internal signals describe the properties of the growing edge, reflecting its characteristic random motion and its stiffness or persistence. The physical interaction with the substrate is encoded in the structure tensor and the local derivatives (gradients) of the surroundings. The structure tensor takes the place of the diffusive tensor in the MGD, while the mean vector is the local gradient . This concept, which is widely used in image analysis (e.g. anisotropic diffusion) [34–36], allows us to include arbitrary structural information by representing it as an image volume and calculating the corresponding structure tensor.
The combination of the different PDFs through convolution (Eq 4) results in a single MGD from which the subsequent growth direction is drawn. This combined PDF distributes random vectors around a mean vector with a length that is the sum of all mean vectors of the individual signals. The shape of this shifted multivariate normal is expressed by its covariance.
Step length
The Multivariate Gaussian Distribution in Eq 3 covers the full , but sampling the entire space to draw the next growth direction would not only be prohibitively slow, but would also generate unphysical step size distributions. This problem can be solved by integrating over the radius to generate a sampling of the PDF from a subset of
(the unit sphere) while retaining the full spatial information contained in its covariance and mean. An additional advantage is that growth direction and step length are separated into two probability distributions which can then be treated independently.
Distributions on the unit sphere can be obtained by embedding
in
. For a random vector
, this could be its conditional distribution on
, or its angular projection
. For the family of Fisher-Bingham distributions [37], the Bingham distribution [38] is the conditional distribution, while the Kent distribution is the projection of the general Fisher-Bingham distribution [39]. Although these distributions are well studied and are commonly used for data fitting and directional statistics [40], the Kent distribution is not suitable for simulations, as it requires a rejection method [41]. Instead, the MGD can be transformed to spherical coordinates, which allows us to calculate a spherical marginal distribution MGDangular by integrating over the radial component to obtain a projection of the MGD. Starting from Eq 3, the k-dimensional coordinate basis is transformed to
with
,
and
. The transformation and integration of Eq 3 results in the following angular MGD:
(7)
Carrying out the integration yields the general angular Gaussian distribution (GAGD), introduced and explicitly calculated for dimensions k = 1, 2, 3 in [42], which is the angular marginal distribution to the MGD that projects the directional probability information onto the -sphere:
(8)
The last term
of Eq 8 is a function
(9)
For the 3-dimensional case k = 3,
, this results in
where ϕ(.) is the standard normal PDF and Φ(.) the corresponding CDF, the cumulative distribution function. For additional detail, please see S1 Text.
This definition separates the radial distribution from the angular distribution. While we use a constant step length in our simulations, this separation would also allow for variable step length PDFs without losing any information of the spatial distribution of external cues with large mean vectors . In Fig 2, we show three distributions on the unit sphere
for three dimensions with different mean vector
and covariance Σ as examples for shifted isotropic signals (persistence), highly anisotropic covariances due to spatial constraints such as edges and walls, and peaked distributions for guidance cues towards specific directions. The structural information is projected onto the unit sphere while retaining the full information about Σ and μ, as shown in Eq 8—e.g., a bimodal MGD still retains a bimodal marginal MGDangular due to its covariance.
Probability distributions are modeled as multivariate Gaussian distributions (MGD), shown in a), with mean and covariance determined from the discrete simulation grid that describes structural and soluble cues. Individual MGDs are combined by convolution, transformed to spherical coordinates, and projected onto the unit sphere (bottom). Examples shown on the right hand side include structural guidance along a plane (e.g. a tissue boundary, b), a unidirectional drift towards the viewer (e.g. a growth factor gradient, c), and persistence due to memory of past growth directions (e.g. bending stiffness, d). The three probability distributions are merged by convolution, restricted by the aperture of the growth cone, and sampled on the corresponding segment of the sphere (e). The next growth direction is drawn from this combined, restricted distribution.
After motivating the mathematical kernel for the growth process and the interaction with the substrate, we now introduce the rules that enable the growing edges to bifurcate, branch off daughter processes, to form new connections (e.g. synapses) with other edges, and to be persistent and locally self-avoiding.
Persistence
During edge outgrowth, the finite bending stiffness of edges causes the growth direction to be correlated to the previous directions, resulting in persistence of the movement. In absence of a drift vector , and since the eigenvectors of the covariance matrix Σ are independent, the random growth process is a Wiener and Markov Process, and still remains a general interpretation of a Wiener process for
. Persistence of the growth direction is equivalent to introducing a drift that is the sum of the last t growth directions
. This results in an effective bending stiffness of the growing edges, and its persistence length is expressed by the Kuhn length [43],
(10)
where δl is the length between two steps and δl〈cosθi,j〉 is the projected length at i towards j (Fig 1). This directional correlation does however not imply self-avoidance within the persistence length scale, as there still is a non-zero probability for an edge to loop back and collide with itself within the persistence length.
Self-avoidance
Local self-avoidance is the phenomenon that growing edges in a biological network, e.g. outgrowing neurites or branching vessels, do not loop back onto themselves [44]. In our growth model, local self-avoidance is achieved by restricting possible growth directions to a solid angle Ω around . The aperture described by Ω is equivalent to the maximum allowed curvature of a growing edge. This means that the heuristic growth process is no longer a Markov process [45], as it loses its reversibility, but now becomes self avoiding within the persistence length. Fig 2 shows the effect of persistence on the angular distribution MGDangular and how the aperture is restricted by Ω.
Connectivity
New nodes in a growing network form by branching or bifurcation of growing edges, or when a growing edge merges with another edge. We distinguish between “branching”, the branching-off of a new edge from a growing edge where the parent edge maintains its growth direction, and “bifurcation”, the splitting of a growing edge into two daughter edges growing in two different directions. Both can occur at every growth step with a probability that depends on edge properties (e.g. age) as well as external cues. Whenever an edge is within a certain distance to another edge, it can merge and form a new node, again with a probability that can depend on edge properties and external signals. Finally, growing edges can terminate and form an end-node, again with a probability that depends on external signals, such as obstacles or the density of already existing edges. Fig 1 schematically shows the time development of a growing network where edges branch several times, merge, and terminate.
Implementation
We next describe the computational implementation of the theoretical framework for spatial network growth described in the previous sections. Key input parameters for the simulation are the spatial environment and the seed points (e.g. cells) that define from where the first edges start to grow. The spatial environment is provided in the form of 3D image volumes that contain information about the physical structure of the growth environment (e.g. tissue structure), as well as about signals that influence the growth process (e.g. growth factors). This grid-based representation of the environment in an otherwise continuous model, as illustrated in Fig 1, makes it possible to study network growth in real tissue environments as measured e.g. with confocal or light sheet microscopy without explicitly stating their geometry by including them as scalar data on a 3D grid or “image” layer. Besides these image data and the seed locations and directions, the simulation requires a number of model parameters such as step length, aperture and degree of persistence as input, listed in Table 2. The aperture is the opening angle of the “field of view” of the growth cone. Smaller values limit the possible directions it can turn to, resulting in self-avoidance as described e.g. for neurons [44]. The angles for branching and bifurcations depend on the underlying biological mechanism, with large angles e.g. for vessel sprouting [46] or small angles for actin-crosslinker mediated branching of cell processes [47]. The corresponding probabilities for branching and bifurcation describe how likely these processes occur in a growing edge, which in biological systems can be both intrinsically determined (e.g. typical rates in osteocyte processes [14]) or externally controlled as in neurite outgrowth [2]. The parameter “memory” sets the number of previous steps that contribute to the next growth direction and determines the persistence of the direction. Biologically, persistent motion can arise either from physical constraints (e.g. persistence length of microtubules) or from the intrinsic persistence of the biological process, e.g. in cell migration. All parameters are set at the beginning of the simulation but can change for each growth cone as a function of time or external signals to capture the corresponding biological mechanism in the simulation.
The PDFs for the growth processes are derived from the image volumes by calculating gradients and the structure tensor (Fig 3). These image volumes can contain multiple channels to describe not only structure, but also scalar and vector fields to describe forces or signaling particles. The spatial growth environment can also be dynamic, either via external time-dependent changes or via changes introduced by the growing network itself. Notably, it is by this reciprocal feedback between the growing network and its spatial environment that the network architecture turns into an emergent property that cannot be encoded in the growth rules alone.
Structural and soluble cues and other signals are provided as 3D image volumes of real or simulated data (left). The six components of the structure tensor define the guidance cues from the substrate structure, whereas the three gradients of the images of soluble cues, e.g. growth factor concentrations, define the resulting directional bias. The feature sizes σi used for the filters determine the resolution of the cues. Other signals can be integrated by computing the corresponding features (filters) that define the resulting growth cue. The image features containing the different cues are Gaussian smoothed and interpolated onto the final simulation grid. The growth simulation uses continuous coordinates on a discrete grid.
The growing edges are implemented as instances of a growth cone class with the properties listed in Table 3. These include both internal parameters as well as functions that define how external cues are translated into growth directions. All object instances work on internal memories, are able to create new objects, and can inherit their parameters. The workflow and interaction of the different memories are summarized in Fig 4.
Starting conditions are provided as image volumes and as list of initial positions and directions. During each time step, the steps in the grey box are performed in parallel for each cone. The four elementary event types during network growth occur with their respective probabilities (A), depending on the local environment. Afterwards, the next position of each cone is determined from the spherical probability distribution computed from the local environment (B). Finally, the computation grid (left) and network dictionary (right), which are kept in shared memory, are updated by each parallel process, such that they can provide up-to-date conditions for all cones in the subsequent step. Finally, the network graph is constructed, its topological properties computed, and the network is visualized. These analysis steps can also be performed periodically during the simulation to monitor network development on-the-fly.
Prior to deciding on the growth direction according to Eq 8 angular, the cone checks if there is a merge, branch, bifurcation or termination event. For merging, the cone at position performs a collision detection with all edges En of an ensemble n ∈ ΩE that are within its search field of radius Rc. ΩE includes all edges of a cluster specified through its members that can interact during t growth steps. A new node is created at the ith positions
of the jth edge j ∈ ΩE that fulfils the condition to be in shortest distance to the cone position.
(11)
The cone terminates afterwards and the memory attributes of all participating objects are updated. Branching and bifurcation are executed via object functions. The events occur randomly with their respective probability. This probability and the parameters for branching angle and direction as well as the bifurcation plane are coupled to the external parameters of the spatial environment. The same type of coupling is implemented for termination and reactivation processes. If an agent branches or bifurcates, a new node is created at the splitting position. In the case of branching, a new cone object emerges from the new edge, and the mother branch updates its memory. For bifurcation, the old cone terminates, and two or more new cone objects are created and grow in new directions with an oblique angle to each other.
As the framework relies heavily on the communication of all objects, an efficient parallelization strategy is important. Already existing edges are not only stored in the graph dictionary, but also generate an imprint in an additional layer of the simulation grid. During growth, each process checks for the presence of existing edges by generating a view in this grid. If there is an edge in the vicinity, it can be looked up in the dictionary using its identifier from the simulation grid. To additionally reduce the number of collision calculations between actively moving cones, a Voronoi tessellation is performed for all growing edges to cluster neighbouring cones within a Delaunay distance ld. Only those agents that are able to interact during the following ld-steps have to communicate during these steps, while all clusters are independent from each other and can be updated in parallel. The clustering step is repeated after ld steps. In Fig 5, a 2-dimensional clustering is shown as an example. All agents with overlapping blue circles can interact and are part of the same cluster.
In (a), a 2-dimensional illustration of the clustering scheme to reduce the number of collision detections is shown for illustration. Blue dots correspond to active growth processes, and only processes with overlapping light-blue circles can interact. A Voronoi tessellation of the simulation volume is performed for all growing edges to cluster all neighbouring cones within a Delaunay distance ld. Only agents that can interact during the following ld-steps have to communicate during these steps, while all clusters are independent from each other and can be updated in parallel. In (b) and (c), the scaling of the simulation time with the number of growth cones and worker processes is shown.
Applications
We used our model to simulate the development of a multicellular biological network in a tissue with different degrees of anisotropy. This scenario is representative for a variety of biological systems where the final network architecture depends on the interplay of genetically encoded cell behaviour and the complex structured tissue environment. Examples include the osteocyte lacuno-canalicular system in bone [48], the development and regeneration of neuronal networks [2], or the reticular cell network in lymph nodes [15]. To show how our framework can be used to study the relationship between local growth rules, tissue structure, and global emergent network architecture, we performed growth simulations for different ranges of branching probability and angle, growth cone aperture, directional memory, as well as for different degrees of tissue anisotropy. To account for the stochastic nature of the growth process, we created a sample of ten random networks with identical starting parameters and calculated mean values. We then quantified the influence of the varying parameters on the resulting node number and average node degree, clustering coefficient, average shortest path length, edge density, as well as three types of node centralities. Closely spaced nodes are combined into a single node to enable comparison with experimentally obtained imaging data with finite spatial resolution [14]. Examples of the resulting networks and parameter relationships are shown in Fig 6, while the results of the sensitivity analysis of global network topology on the growth parameters are shown in Table 4. As can be seen in Fig 6A, the average node degree of the network gradually increases during the development of the network. A giant component emerges as all cells become connected to each other, indicating a critical point after around 30 steps. With increasing branching probability, the network becomes more dense, with higher mean degree and clustering coefficient and reduced average shortest path length (Fig 6B). Average node betweenness and information centrality decrease while harmonic centrality increases with higher branching rate. All three centralities approach the experimentally measured values in osteocyte networks [49]. We also find that stronger tissue anisotropy leads to a lower number of nodes but increasing clustering coefficient and average shortest path length, while the average node degree remains unchanged (Fig 6C). These observations suggest that the ability of growing osteocyte processes to branch at a sufficiently high rate together with a sufficient degree of tissue organization are important parameters for the formation of viable osteocyte networks in bone. This example outlines the possibility for more systematic studies, e.g. to identify the underlying molecular and cellular parameters for observed changes in network architecture in different biological systems. A similar comparison could be performed for any other 3D biological network for which such quantitative data are available. By modifying the provided code used to generate Fig 6 and Table 4, these examples can easily be adapted to other biological systems.
a) Cells (blue) emanate growth processes that form a dense, interconnected network of active (green) and terminated (red) edges. Node degree goes up (top) and a giant component emerges that connects all cells (bottom), as evident in the connectivity graphs (red/black). b) Higher branching probability leads to increasing clustering coefficient and shorter average paths. Node betweenness and harmonic centrality approach experimental values of osteocyte networks in bone (blue). c) Stronger tissue anisotropy leads to smaller networks with increased clustering coefficient and shortest path length, and decreased information centrality, approaching experimental values (blue).
Discussion
We developed a novel agent-based framework to simulate the growth of networks in three-dimensional space. The outgrowth of individual processes follows a 3D random walk with memory (correlation) and directional bias due to interactions with the environment. This framework can be used to simulate the growth of biological networks on all scales from single cells to ecosystems, or of generic network development through interaction of autonomous agents with each other and with external physical and chemical cues. During simulation, the network is directly converted into a graph using the networkx library [50], which provides access to many quantitative and comparative measures of network topology.
The first agent-based computer simulation of 2D spatial network growth, published in 1967 [20], already used a similar approach as our model: edges grow and branch according to rules, respond to attractive and repulsive cues, and follow density gradients. The next growth direction is randomly assigned based on sampling the space around the growth tip. Inspired by plant growth, this model can generate a variety of tree-like networks—a remarkable achievement given the limitations of computer simulations more than 50 years ago. While the field of complex networks flourished decades later, relatively little work was done on spatial network growth [1, 19]. In 2009, two new computational frameworks for neuronal network growth were published, CX3D [25] and NETMORPH [26]. Both use biologically realistic growing and branching rules and are openly available. In both models, synapse formation only takes place after the growth phase. CX3D offers a more physically realistic treatment of mechanical forces compared to our model, but lacks the flexibility of grid-based external structural cues. Two other neuron growth models [23, 51] efficiently generate neuron tree morphologies and space-filling networks, but without biologically realistic growth mechanisms and synapse formation. Taylor-King et al. [52] use an elegant mean field approach for network growth based on local state degree distributions. Whenever it is biologically plausible to analytically formulate local rules, such an approach can accurately reproduce global properties. In their model for vessel tree development, Perfahl et al. [53] include physical interactions with the environment and between edges to study the role of mechanical forces for vessel development. In their “Unifying theory of branching morphogenesis”, Hannezo et al. [28] present a stochastic model for epithelial duct development based on branching and annihilating random walks. Although interactions are limited to explicitly defined chemical gradients and anisotropy, this model can reproduce experimental tree topologies for a variety of organs. We summarized this comparison of our model with prior work in S1 Table. To our knowledge, our work presents the first framework for network growth that includes a biologically motivated local growth process as well as arbitrary interactions with an external environment, and is fully available as computational implementation.
We did not include energy consumption in our model in order to keep it as generic as possible. Depending on the type of network, different mechanisms of energy sources, energy distribution and consumption are possible. As an example, energy could either be “harvested” from the environment by the growing processes, or distributed throughout the network from cell bodies. Other existing models either omit energy consumption as well, or contain only specific energy-related mechanisms such as competition between growth cones [26] or diffusion of proteins from the soma [25].
One class of biological networks where the interaction and feedback between invidiual growth cones and the microenvironment during growth determines the resulting network architecture are multicellular networks, such as neuronal networks in the brain or the osteocyte network in bone. In bone, different degrees of tissue organization and the presence of soluble cues determines the connectivity and arrangement of the resulting network. With the framework presented here, the question can be investigated how local cellular response and resulting network topology depend on each other by conducting virtual experiments with varying parameters, which would not be possible in real experiments. The resulting theoretical understanding can then inform the design of guiding structures (scaffolds) to facilitate optimal network development in regenerating bone by providing the relevant physical and chemical signals [54].
The growth of vascular and neuronal cell networks during embryonal development is another example where soluble cues, the geometry of the environment, and the interaction between nearby cells all play an important role [55]. Recent improvements of lightsheet imaging technology now allow to visualize the entire growth process with high spatial and temporal resolution [56]. Our framework predicts not only the outcome, but also the dynamics of this process, which makes it possible to test specific hypotheses about the biological mechanisms that locally control this growth process. In the future, this might contribute to solving the question to what degree functional network architecture is encoded in the genome, or how the interactions and mechanochemical feedback loops between cells and the environment during growth determine the resulting tissue organization. Another interesting example is the role of curvature for tissue growth and organization [57]. With our framework, it will be possible to systematically vary the curvature of growth surfaces for networks while including other relevant biological and physical signals in the same simulation framework.
A very interesting related field is the exploratory growth of plants guided by pairwise interactions, tropic reponses to signals (e.g. light), and nastic (e.g. helical) movements. Recently, a computational model was introduced to model the dynamics of such sensory-growth systems [58, 59], taking into account the mechanical properties of the growing system. Such a framework offers interesting opportunities not only to understand biological control principles, but also for designing self-growing artificial systems. It would be interesting to see how this approach can be extended towards the development of functional network architectures.
While we developed our framework with network growth in mind, it can also be turned around and used to find and trace network structures in noisy image volumes by treating them as guiding signals for growth. Many approaches exist to trace filaments [60, 61], but they often have difficulties with branching structures. By simulating multiple growth processes either directly on image data or e.g. on probability maps predicted by convolutional neural networks, our framework might be able to find also highly branching and irregular structures.
Network growth as defined here is a parallel process by definition and thus can be computed in a distributed manner. We describe an efficient strategy for parallelization while taking into account the interaction between neighbouring growth cones. Future improvements could make use of the massively parallel processing power of graphical processing units.
Conclusion
We developed a probabilistic agent-based model to describe individual growth processes as biased, correlated random motion with rules for branching, bifurcation, merging and termination. Using directional statistics, the influence of external cues on the growth process could be obtained directly from real image data. Our tool allows us to study the relationship between biological growth parameters and macroscopic function, and can generate plausible network graphs that take local feedback into account as basis for comparative studies using graph-based methods. Our approach is not limited to a specific type of network, includes a thorough treatment of probabilities, and can easily include arbitrary external constraints. We implemented our model in python and make it freely available as a computational framework, that allows other researchers to perform simulations for diverse types of biological networks on all scales.
Methods
All edges and nodes are instances of their respective classes, with the attributes shown in Table 3. The framework was implemented in Python 3 using numpy [62, 63]. The network is directly converted into a networkx graph for further processing and analysis. On a 8-core Intel Xeon 6134 3.2 GHz machine with 64 GB RAM, running a network simulation with 100 growth cones in a 256x256x256 tissue volume for 50 steps takes 60 seconds. The scaling of the simulation time with the number of growth cones and worker processes is shown in Fig 5. Source code, documentation and example configurations of our computational model are available at https://github.com/CIA-CCTB/pythrahyper_net.
Supporting information
S1 Text. Detailed description of 3D random walks in complex environments.
https://doi.org/10.1371/journal.pcbi.1008003.s001
(PDF)
S1 Table. Extended version of Table 1 with different external signals and cues.
https://doi.org/10.1371/journal.pcbi.1008003.s002
(PDF)
S2 Table. Comparison of our framework with prior work.
https://doi.org/10.1371/journal.pcbi.1008003.s003
(PDF)
References
- 1.
Barthélemy M. Morphogenesis of Spatial Networks. Lecture Notes in Morphogenesis. Springer International Publishing; 2018. Available from: https://www.springer.com/de/book/9783319205649.
- 2. Kaiser M. Mechanisms of Connectome Development. Trends in Cognitive Sciences. 2017 Sep;21(9):703–717.
- 3. Maskery S, Shinbrot T. Deterministic and stochastic elements of axonal guidance. Annual Review of Biomedical Engineering. 2005;7:187–221.
- 4. Chédotal A, Richards LJ. Wiring the Brain: The Biology of Neuronal Guidance. Cold Spring Harbor Perspectives in Biology. 2010 Jun;2(6). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2869517/. pmid:20463002
- 5. Banerjee S, Gardel ML, Schwarz US. The Actin Cytoskeleton as an Active Adaptive Material. Annual Review of Condensed Matter Physics. 2020;11(1):421–439. Available from: https://doi.org/10.1146/annurev-conmatphys-031218-013231.
- 6. Secomb TW, Alberding JP, Hsu R, Dewhirst MW, Pries AR. Angiogenesis: An Adaptive Dynamic Biological Patterning Problem. PLOS Computational Biology. 2013 Mar;9(3):e1002983. Available from: http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002983. pmid:23555218
- 7. Kerschnitzki M, Kollmannsberger P, Burghammer M, Duda GN, Weinkamer R, Wagermaier W, et al. Architecture of the osteocyte network correlates with bone material quality. Journal of Bone and Mineral Research: The Official Journal of the American Society for Bone and Mineral Research. 2013 Aug;28(8):1837–1845. pmid:23494896
- 8. Jones J. Characteristics of pattern formation and evolution in approximations of Physarum transport networks. Artificial Life. 2010;16(2):127–153.
- 9. Bebber DP, Hynes J, Darrah PR, Boddy L, Fricker MD. Biological solutions to transport network design. Proceedings Biological Sciences. 2007 Sep;274(1623):2307–2315.
- 10. Duran-Nebreda S, Bassel GW. Bridging Scales in Plant Biology Using Network Science. Trends in Plant Science. 2017 Dec;22(12):1001–1003. Publisher: Elsevier. Available from: https://www.cell.com/trends/plant-science/abstract/S1360-1385(17)30216-9. pmid:29029828
- 11. Eichler K, Li F, Litwin-Kumar A, Park Y, Andrade I, Schneider-Mizell CM, et al. The complete connectome of a learning and memory centre in an insect brain. Nature. 2017 Aug;548(7666):175–182. Available from: https://www.nature.com/articles/nature23455. pmid:28796202
- 12. Meinertzhagen IA. Of what use is connectomics? A personal perspective on the Drosophila connectome. Journal of Experimental Biology. 2018 May;221(10):jeb164954. Available from: http://jeb.biologists.org/content/221/10/jeb164954. pmid:29784759
- 13. Yan G, Vértes PE, Towlson EK, Chew YL, Walker DS, Schafer WR, et al. Network control principles predict neuron function in the Caenorhabditis elegans connectome. Nature. 2017 Oct;advance online publication. Available from: http://www.nature.com/nature/journal/vaop/ncurrent/full/nature24056.html?WT.ec_id=NATURE-20171019&spMailingID=55167568&spUserID=OTMzMjIzOTczNQS2&spJobID=1263146646&spReportId=MTI2MzE0NjY0NgS2. pmid:29045391
- 14. Buenzli PR, Sims NA. Quantifying the osteocyte network in the human skeleton. Bone. 2015 Jun;75:144–150. Available from: http://www.sciencedirect.com/science/article/pii/S8756328215000617. pmid:25708054
- 15. Novkovic M, Onder L, Bocharov G, Ludewig B. Topological Structure and Robustness of the Lymph Node Conduit System. Cell Reports. 2020 Jan;30(3):893–904.e6. Available from: https://www.cell.com/cell-reports/abstract/S2211-1247(19)31727-9. pmid:31968261
- 16. Erdös P, Rényi A. On random graphs. I. Publicationes Mathematicae Debrecen. 1959;6:290–297. Available from: https://mathscinet.ams.org/mathscinet-getitem?mr=0120167.
- 17. Watts DJ, Strogatz SH. Collective dynamics of ‘small-world’ networks. Nature. 1998 Jun;393:440–442. Available from: http://adsabs.harvard.edu/abs/1998Natur.393..440W. pmid:9623998
- 18. Comin C, da F Costa L. Spatial Networks: When Topology Meets Geometry (CDT-3). ResearchGate; 2018.
- 19. Kaiser M, Hilgetag CC. Spatial growth of real-world networks. Physical Review E. 2004 Mar;69(3):036103. Available from: https://link.aps.org/doi/10.1103/PhysRevE.69.036103. pmid:15089357
- 20. Cohen D. Computer Simulation of Biological Pattern Generation Processes. Nature. 1967 Oct;216(5112):246–248. Available from: https://www.nature.com/articles/216246a0.
- 21. Zitin A, Gorowara A, Squires S, Herrera M, Antonsen TM, Girvan M, et al. Spatially embedded growing small-world networks. Scientific Reports. 2014 Nov;4:7047. Available from: https://www.nature.com/articles/srep07047. pmid:25395180
- 22. Nicosia V, Vértes PE, Schafer WR, Latora V, Bullmore ET. Phase transition in the economically modeled growth of a cellular nervous system. Proceedings of the National Academy of Sciences. 2013 May;110(19):7880–7885. Available from: http://www.pnas.org/content/110/19/7880. pmid:23610428
- 23. Cuntz H, Forstner F, Borst A, Häusser M. One Rule to Grow Them All: A General Theory of Neuronal Branching and Its Practical Application. PLOS Computational Biology. 2010 May;6(8):e1000877. Available from: http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000877. pmid:20700495
- 24. Ronellenfitsch H, Katifori E. Global Optimization, Local Adaptation, and the Role of Growth in Distribution Networks. Physical Review Letters. 2016 Sep;117(13):138301. ArXiv: 1606.00331. Available from: http://arxiv.org/abs/1606.00331. pmid:27715085
- 25. Zubler F, Douglas R. A framework for modeling the growth and development of neurons and networks. Frontiers in Computational Neuroscience. 2009;3. Publisher: Frontiers. Available from: https://www.frontiersin.org/articles/10.3389/neuro.10.025.2009/full. pmid:19949465
- 26. Koene RA, Tijms B, van Hees P, Postma F, de Ridder A, Ramakers GJA, et al. NETMORPH: A Framework for the Stochastic Generation of Large Scale Neuronal Networks With Realistic Neuron Morphologies. Neuroinformatics. 2009 Sep;7(3):195–210. Available from: https://doi.org/10.1007/s12021-009-9052-3. pmid:19672726
- 27. Gaiselmann G, Thiedmann R, Manke I, Lehnert W, Schmidt V. Stochastic 3D modeling of fiber-based materials. Computational Materials Science. 2012 Jun;59:75–86. Available from: http://www.sciencedirect.com/science/article/pii/S0927025612001267.
- 28. Hannezo E, Scheele CLGJ, Moad M, Drogo N, Heer R, Sampogna RV, et al. A Unifying Theory of Branching Morphogenesis. Cell. 2017 Sep;171(1):242–255.e27. Available from: http://www.sciencedirect.com/science/article/pii/S0092867417309510. pmid:28938116
- 29. Codling EA, Plank MJ, Benhamou S. Random walk models in biology. Journal of the Royal Society Interface. 2008 Aug;5(25):813–834. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2504494/. pmid:18426776
- 30.
Risken H, Frank T. The Fokker-Planck Equation: Methods of Solution and Applications. 2nd ed. Springer Series in Synergetics. Berlin Heidelberg: Springer-Verlag; 1996. Available from: https://www.springer.com/de/book/9783540615309.
- 31. Lemons DS, Gythiel A. Paul Langevin’s 1908 paper “On the Theory of Brownian Motion” [“Sur la théorie du mouvement brownien,” C. R. Acad. Sci. (Paris) 146, 530–533 (1908)]. American Journal of Physics. 1997 Nov;65(11):1079–1081. Available from: https://aapt.scitation.org/doi/10.1119/1.18725.
- 32. Patlak CS. Random walk with persistence and external bias. The bulletin of mathematical biophysics. 1953 Sep;15(3):311–338. Available from: https://doi.org/10.1007/BF02476407.
- 33. Boas ML, Romain JE. Mathematical Methods in the Physical Sciences. Physics Today. 2009 Jan;20(6):81. Publisher: American Institute of PhysicsAIP. Available from: https://physicstoday.scitation.org/doi/abs/10.1063/1.3034374.
- 34. Koenderink JJ. The structure of images. Biological Cybernetics. 1984 Aug;50(5):363–370. Available from: https://doi.org/10.1007/BF00336961.pmid:6477978
- 35. TschumperlÉ D. Fast Anisotropic Smoothing of Multi-Valued Images using Curvature-Preserving PDE’s. International Journal of Computer Vision. 2006 Jun;68(1):65–82. Available from: https://doi.org/10.1007/s11263-006-5631-z.
- 36.
Weickert J. A review of nonlinear diffusion filtering. In: ter Haar Romeny B, Florack L, Koenderink J, Viergever M, editors. Scale-Space Theory in Computer Vision. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer; 1997. p. 1–28.
- 37.
Fisher NI, Lewis T, Embleton BJJ. Statistical Analysis of Spherical Data. Cambridge: Cambridge University Press; 1987. Available from: https://www.cambridge.org/core/books/statistical-analysis-of-spherical-data/CCFE72380FB730E48080D7596135A363.
- 38. Bingham C. An Antipodally Symmetric Distribution on the Sphere. The Annals of Statistics. 1974;2(6):1201–1225. Available from: https://www.jstor.org/stable/2958339.
- 39. Kent JT. The Fisher-Bingham Distribution on the Sphere. Journal of the Royal Statistical Society Series B (Methodological). 1982;44(1):71–80. Available from: https://www.jstor.org/stable/2984712.
- 40.
Mardia KV, Jupp PE. Directional Statistics. John Wiley & Sons; 2009.
- 41.
Kent JT, Ganeiber AM, Mardia KV. A new method to simulate the Bingham and related distributions in directional data analysis with applications. arXiv:13108110 [math, stat]. 2013 Oct;ArXiv: 1310.8110. Available from: http://arxiv.org/abs/1310.8110.
- 42. Paine PJ, Preston SP, Tsagris M, Wood ATA. An elliptically symmetric angular Gaussian distribution. Statistics and Computing. 2018 May;28(3):689–697. Available from: https://doi.org/10.1007/s11222-017-9756-4.
- 43. Kuhn W. Über die Gestalt fadenförmiger Moleküle in Lösungen. Kolloid-Zeitschrift. 1934 Feb;68(1):2–15. Available from: https://doi.org/10.1007/BF01451681.
- 44. Long H, Ou Y, Rao Y, Meyel DJv. Dendrite branching and self-avoidance are controlled by Turtle, a conserved IgSF protein in Drosophila. Development. 2009 Oct;136(20):3475–3484. Publisher: The Company of Biologists Ltd Section: Research Article. Available from: https://dev.biologists.org/content/136/20/3475. pmid:19783736
- 45.
Takács L. Non-Markovian Processes. In: Takács L, editor. Stochastic Process: Problems and Solutions. Methuen’s Monographs on Applied Probability and Statistics. Dordrecht: Springer Netherlands; 1966. p. 46–62. Available from: https://doi.org/10.1007/978-94-011-6020-9_4.
- 46. Chappell JC, Wiley DM, Bautch VL. How Blood Vessel Networks Are Made and Measured. Cells Tissues Organs. 2012;195(1-2):94–107. Publisher: Karger Publishers. Available from: https://www.karger.com/Article/FullText/331398. pmid:21996655
- 47. Lanoue V, Cooper HM. Branching mechanisms shaping dendrite architecture. Developmental Biology. 2019 Jul;451(1):16–24. Available from: http://www.sciencedirect.com/science/article/pii/S0012160618303373. pmid:30550882
- 48. Bonewald LF. The amazing osteocyte. Journal of Bone and Mineral Research: The Official Journal of the American Society for Bone and Mineral Research. 2011 Feb;26(2):229–238.
- 49. Kollmannsberger P, Kerschnitzki M, Repp F, Wagermaier W, Weinkamer R, Peter Fratzl. The small world of osteocytes: connectomics of the lacuno-canalicular network in bone. New Journal of Physics. 2017;19(7):073019. Available from: http://stacks.iop.org/1367-2630/19/i=7/a=073019.
- 50.
Hagberg AA, Schult DA, Swart PJ. Exploring Network Structure, Dynamics, and Function using NetworkX. Proceedings of the 7th Python in Science Conference. 2008;p. 11–15.
- 51. Vanherpe L, Kanari L, Atenekeng G, Palacios J, Shillcock J. Framework for efficient synthesis of spatially embedded morphologies. Physical Review E. 2016 Aug;94(2):023315. Available from: https://link.aps.org/doi/10.1103/PhysRevE.94.023315. pmid:27627420
- 52. Taylor-King JP, Basanta D, Chapman SJ, Porter MA. Mean-field approach to evolving spatial networks, with an application to osteocyte network formation. Physical Review E. 2017 Jul;96(1):012301. Available from: https://link.aps.org/doi/10.1103/PhysRevE.96.012301. pmid:29347066
- 53. Perfahl H, Hughes BD, Alarcón T, Maini PK, Lloyd MC, Reuss M, et al. 3D hybrid modelling of vascular network formation. Journal of Theoretical Biology. 2017 Feb;414(Supplement C):254–268. Available from: http://www.sciencedirect.com/science/article/pii/S0022519316303782. pmid:27890575
- 54. Bidan CM, Kommareddy KP, Rumpler M, Kollmannsberger P, Fratzl P, Dunlop JWC. Geometry as a Factor for Tissue Growth: Towards Shape Optimization of Tissue Engineering Scaffolds. Advanced Healthcare Materials. 2013;2(1):186–194. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1002/adhm.201200159. pmid:23184876
- 55. Roy S. Development of the zebrafish nervous system: Mechanisms of cellfate specification and axonal pathfinding in the central nervous system and periphery. Current Science. 1994;66(9):629–633. Publisher: Current Science Association. Available from: https://www.jstor.org/stable/24098781.
- 56. Huisken J, Swoger J, Bene FD, Wittbrodt J, Stelzer EHK. Optical Sectioning Deep Inside Live Embryos by Selective Plane Illumination Microscopy. Science. 2004 Aug;305(5686):1007–1009. Publisher: American Association for the Advancement of Science Section: Report. Available from: https://science.sciencemag.org/content/305/5686/1007. pmid:15310904
- 57. Kollmannsberger P, Bidan CM, Dunlop JWC, Fratzl P. The physics of tissue patterning and extracellular matrix organisation: how cells join forces. Soft Matter. 2011 Oct;7(20):9549–9560. Publisher: The Royal Society of Chemistry. Available from: https://pubs.rsc.org/en/content/articlelanding/2011/sm/c1sm05588g.
- 58. Bastien R, Porat A, Meroz Y. Towards a framework for collective behavior in growth-driven systems, based on plant-inspired allotropic pairwise interactions. Bioinspiration & Biomimetics. 2019 Aug;14(5):055004. Available from: https://iopscience.iop.org/article/10.1088/1748-3190/ab30d3/meta. pmid:31292284
- 59. Porat A, Tedone F, Palladino M, Marcati P, Meroz Y. A general 3D model for growth dynamics of sensory-growth systems: from plants to robotics. bioRxiv. 2020 Apr;p. 2020.04.21.053033. Publisher: Cold Spring Harbor Laboratory Section: New Results. Available from: https://www.biorxiv.org/content/10.1101/2020.04.21.053033v1.
- 60. Türetken E, Benmansour F, Andres B, Głowacki P, Pfister H, Fua P. Reconstructing Curvilinear Networks Using Path Classifiers and Integer Programming. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2016 Dec;38(12):2515–2530.
- 61. Acciai L, Soda P, Iannello G. Automated Neuron Tracing Methods: An Updated Account. Neuroinformatics. 2016 Oct;14(4):353–367. Available from: https://doi.org/10.1007/s12021-016-9310-0. pmid:27447185
- 62. Virtanen P, Gommers R, Oliphant TE, Haberland M, Reddy T, Cournapeau D, et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature Methods. 2020 Mar;17(3):261–272. Number: 3 Publisher: Nature Publishing Group. Available from: https://www.nature.com/articles/s41592-019-0686-2. pmid:32015543
- 63.
van der Walt S, Colbert SC, Varoquaux G. The NumPy Array: A Structure for Efficient Numerical Computation. Computing in Science Engineering. 2011 Mar;13(2):22–30. Conference Name: Computing in Science Engineering.