Case study: A system with N = 8 excitable units

We consider sequences of seven learned patterns ξ¹, …, ξ⁷ (named ABCDEFG) encoded by eight units x₁, …, x₈. The sequence represents the sequential activation of pairs of units 1-2, 2-3, 3-4, 4-5, 5-6, 6-7 and 7-8, corresponding to patterns A and B, B and C, etc. with an overlap of one unit between them (see Fig 1a). Learning is reported to rely on changes in the efficacy of the synapses between neurons [69] through long term potentiation (LTP) and long term depression (LTD) [70–72]. As a consequence, LTP/LTD potentiates/depresses synapses between units coding for patterns as a function of their overlap, that is synapses between units coding for overlapping patterns are more potentiated. Due to the constant overlap, all synapses between overlapping patterns are equal. Note that the matrix J^max is learned as a function of the overlap between patterns without imposing any sequences. A consequence is that learning of independent pairs of patterns generates a matrix that allows for the activation of sequences.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Directional sequences of an endpoint stimulus-driven system.

(a) Each numbered circle represents a unit. Consecutive units encode a pattern. Except for x₁ and x₈, each unit participates in two patterns. A forward sequence is the activation of units in increasing order. A backward sequence is the activation of units in decreasing order. (b) Left panel: System initialised from the pattern A follows the forward sequence until the pattern F. Right panel: System initialised from the pattern G follows the backward sequence until the pattern B. Same colour code is used to represent units’ indices in (a) and (b). Parameters: μ = 0.41, λ = 0.51, I = 0, ρ = 1.8, τ_r = 900, η = 0.02.

https://doi.org/10.1371/journal.pone.0231165.g001

A system of N = 8 excitatory units can encode P = N − 1 = 7 regular patterns in J^max (see Fig 1a). Encoded memory items can be retrieved either spontaneously (in a noisy environment) or when the memory network is triggered by an external cue [42, 73]. Units x₁ and x₈ are the least self-excited units with , thus it is very unlikely to active them unless they are part of the initial activity state. Hence, the longest chain has P − 1 = 6 consecutive patterns.

Directional sequences from a stimulus-driven pattern in the sequence.

Starting from the first pattern A, the directional activation corresponds to the sequence ABCDEFG (Fig 1b left panel). The forward direction is imposed by J^max because x₁ is less excited since . Hence, while the synaptic variables s₁ and s₂ are equal and decreasing together as the system lies in the vicinity of ξ¹, x₁ is deactivated before x₂. In the same interval of time s₂ < s₃ and s₂ − s₃ increases so that x₂ becomes unstable before x₃ and the system now may converge to ξ². The process can be repeated between ξ² and ξ³ etc. Similarly, starting from the last pattern G gives the reverse direction (GFEDCBA) to the system (Fig 1b right panel).

Initialising the system from a middle pattern ξⁱ does not introduce any direction, since the two active units of ξⁱ are equally excited. While their synaptic variables are decreasing together, depending on the noise at the moment when ξⁱ becomes unstable, either ξⁱ⁻¹ or ξⁱ⁺¹ is activated with equal probabilities. Fig 2 shows the response of the system starting from a mid-point pattern D. The activated sequence can go in either direction DEFG or its reverse DCBA. The random choice for a sequence is driven by a bias in the noise at the time of stimulus-driven activation of the mid-point pattern.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Directional sequences of a midpoint stimulus-driven system.

(a) Each numbered circle represents a unit. Consecutive neurons encode a pattern. Except for x₁ and x₈, each unit participates in the encoding for two patterns. When the system initialised from the pattern D, it follows either the “DCBA” or “DEFG” sequence. (b) Left panel: System initialised from the pattern D follows the “DCBA” sequence until the pattern B. Right panel: System initialised from the pattern D follows “DEFG” sequence until the pattern F. The same colour code is used to represent units’ indices in (a) and (b). Parameters: μ = 0.414, λ = 0.51, I = 0, ρ = 1.8, τ_r = 900, η = 0.02.

https://doi.org/10.1371/journal.pone.0231165.g002

Noise-driven random sequence from a mid-point pattern in the sequence.

The units that participate in two patterns (overlapping units) have stronger self-excitation as it is manifested by the diagonals of J^max. These units (x_i, i ≠ {1, 8}) are likely to be excited by random noise and they can activate others which they encode a pattern with. After a pattern ξⁱ or the associated intermediate pattern being randomly excited by noise, the system can follow either ξⁱ⁻¹ or ξⁱ⁺¹. The robustness of activity depends on the system parameters. Fig 3 shows an example of spontaneous activation of a mid-point pattern D where the directional oriented sequence can be either DEFG or its reverse DCBA. Similar to the system initialised from a middle pattern, the random choice for a direction is driven by a bias in the noise at the time of noise-driven activation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Directional sequences of a spontaneously activated system.

System activated spontaneous by random noise can move in backward (a) or forward (b) directions. Parameters: μ = 0.21, λ = 0.51, I = 0, ρ = 1.8, τ_r = 300, η = 0.04.

https://doi.org/10.1371/journal.pone.0231165.g003

Sensitivity of the dynamics upon parameter values.

We have seen that patterns can be retrieved sequentially when the system is triggered by a cue or spontaneously by noise. However the effectiveness of this process depends on the values of the parameters in Eqs (1) and (2). The dynamics of the system can follow part of the sequence, then either terminate on one pattern ξⁱ with i < N − 1, or converge to a non learned pattern. Moreover we identified two different dynamical scenarios by which a sequence can be followed, depending mainly on the value of μ. This will be analyzed in Sec. Analysis of the dynamics. Here we comment on numerical simulations which highlight the dependency of the sequences upon parameter values.

The behavior of the model was tested on simulation data by measuring the length of regular sequences generated by the network (chain length) and by computing the distance made from the initial pattern after irregular sequences (distance). Chain length and distance were analyzed by fitting linear mixed-effect models (LMM) to the data, using the lmer function from the lme4 package (Version 1.1–7) in R (Version R-3.1.3 [74]). All predictor parameters (inverse of the gain μ, inhibition λ, time constant τ_r, synaptic constant ρ and noise η) were defined as continuous variables and they were centered on their mean. The optimal structure was determined after comparing the goodness of fit of a range of models, using Akaike’s information criterion (AIC); the model with the smallest AIC was selected, corresponding to the model with main effects and interactions between all of the parameters. The significance of the effects was tested using the lmerTest package. For the sake of clarity of the text, we flag the levels of significance with one star (*) if p-value <0.05, two stars (**) if p-value <0.01, three stars (***) if p-value<0.001.

Fig 4 shows time series of the full or partial completion of sequences of retrievals (for N = 8 units) for two different values of noise amplitude η = 0.02 (first row) and η = 0.04 (second and third row). In each case the two first columns show time series with the STD parameters ρ = 1.8 and τ_r = 300 while the last column corresponds to the choice ρ = 1.8 and τ_r = 900. By fixing the synaptic constant ρ = 1.8, we ensure that the synaptic variables s_i decay to the same value S with a decay time depending on τ_r (see Sec. Model). The global inhibition coefficient λ is set at 0.51 in rows Fig 4a and 4b and λ = 0.56 in row Fig 4c. For each choice of the STD parameters μ takes two values, either μ = 0.41 or μ = 0.21.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Response of the system initialised from pattern A (units 1-2) to different levels of noise η and system parameters λ, μ, τ_r.

Synaptic variables are faster along the first two columns (τ_r = 300) than the last column (τ_r = 900). In all simulations I = 0. Row (a) η = 0.02, λ = 0.51. The system with fast synapses can follow the longest sequence from A (units 1-2) to F (units 6-7) for μ = 0.41 (the first panel) but not for μ = 0.21 (the second panel), while the slow synapses can trigger the longest sequence (the third panel). Row (b) η = 0.04, λ = 0.51. Increasing the noise amplitude enables the activation of the whole sequence with fast synapses (the first two panels), whereas the slow synapses give either very short patterns or 3 co-active units, the third panel, respectively. Row (c) η = 0.04, λ = 0.56. Increasing the global inhibition λ regulates the transition for slow synapses (the third panel), whereas the system with fast synapses and μ = 0.41 (the first panel) randomly activates learned patterns and yields short regular and irregular sequences. On the other hand, the system with μ = 0.21 and fast synapses (the second panel) can preserve a regular sequence.

https://doi.org/10.1371/journal.pone.0231165.g004

Observe that the sequence and the pattern durations are shorter in the system with fast synapses (τ_r = 300) than the one with slow synapses (τ_r = 900). In the case of weaker noise (Fig 4a) and fast synapses, the system follows the sequence ABCDEF when μ = 0.41 whereas it stops at the pattern B when μ = 0.21. In other words, increasing μ (decreasing neural gain) in the system with fast synapses recruits more units sequentially. Another way to increase the chain length for μ = 0.21 is slowing down the synaptic variables. The system with slow synapses can follow the sequence ABCDEF for a wide range of μ values. In fact the two different values of μ in Fig 4 correspond to the two different dynamical scenarios which have been evoked in the beginning of this section. This point will be developed in Sec. Analysis of the dynamics. When noise is stronger (Fig 4b) the picture is different: the full sequence can be completed with fast synapses even for μ = 0.21. However, the sequence is shorter with slow synapses and the system quickly explores unexpected patterns like one with three excited units 2, 3, 4 around t = 600 (which is not a learned pattern) in third panel of Fig 4b. These type of activity can be observed for a wide range of μ values with slow synapses.

Comparison between Fig 4b and 4c exemplifies the effect of changing inhibition λ and μ (inverse of neural gain) for the same noise amplitude. Increasing the inhibition coefficient λ regulates the transition for slow synapses, while fast synapses and high values of μ (low neural gain) randomly activates the learned patterns and yields short sequences. The latter is also due to the self inhibition in the system given by the −μx_i term in Eq (1) which facilitates deactivation of an active unit, but makes difficult for an inactive unit to be activated if it is too high. Notice the self inhibitory effect in the fast synapses can be compensated for slow synapses and small μ (high neural gain) which can regularize sequences.

Length of a chain.

When the patterns in a chain are explored in the right order by the system we call it regular. As we saw in Sec. Sensitivity of the dynamics upon parameter values it can happen that only part of the full regular chain has been realised before it stops or starts exploring patterns in a different order, hence activating an irregular chain. We call the partial regular chain a regular segment and its length is the number of patterns it contains. Here we investigate the maximal length that a regular segment starting at pattern A can attain. This length is the rank of the last activated pattern over simulations. It depends on noise η but also on the neuronal parameters (μ, λ) and on the synaptic parameters (τ_r, ρ). As it can be read in Eq (3), ρ characterizes the limiting decay state of the synaptic variable. Possible impact of τ_r and ρ on the dynamics has been investigated in [40]. It has been shown in a system with a a structured connectivity matrix that deactivation of a unit is harder for ρ being small while too high ρ prevents recruiting new units. Thus, ρ determines the reliability of a sequential activation. On the other hand, the threshold of the noise need for activation of a chain decreases with increasing τ_r for ρ constant.

The relation between the model parameters and the sequential activation is more subtle with the learning matrix (4) derived from the most simplified Hebbian learning rule than a structured one as in [40]. In Figs 5 and 6 we present the mean chain length for two different noise intensities; η = 0.02 and η = 0.04, respectively. In each figure synaptic parameters are ρ ∈ {1.2, 2.4},τ_r ∈ {300, 900}. Neural parameters λ and μ are varied within a range assuring the existence of chains of at least length 2. Details of the sequences are shown in S1 and S2 Figs. Statistics on chain length show main effects of each parameter (η (***), μ (**), λ (***), τ_r (***) and ρ (***). They also show an interaction between the five parameters altogether (***).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Average chain length in a regular segment for noise η = 0.02.

Synaptic time constant equals to τ_r = 300 on panels (a) and (c), and τ_r = 900 on panels (b) and (d). (a, b) Activity for ρ = 1.2. The chain length increases with μ (1/neural gain) and τ_r. The global inhibition value, λ, should be high enough for a sequential activation, but the chain length decreases if λ is too high. (c, d) Activity for ρ = 2.4. The chain length increases with μ and τ_r, but decreases with λ. See S1 Fig for the details of the activity.

https://doi.org/10.1371/journal.pone.0231165.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Average chain length in a regular segment for noise η = 0.04.

Synaptic time constant equals to τ_r = 300 on panels (a) and (c), and τ_r = 900 on panels (b) and (d). (a, b) Activity for ρ = 1.2. The chain length increases with τ_r. Chains are longer for intermediate values of μ (intermediate values of neural gain), but shorter for μ too high (low neural gain). Increasing inhibition λ facilitates regular pattern activation for the low values of μ, see for instance λ = 0.501 vs λ = 0.601 in (a), but ceases if it is too strong. (c, d) Activity for ρ = 2.4. The chain length increases with τ_r, but decreases with λ. Increasing μ lengthens the chains more with τ_r = 900 than τ_r = 300. See S2 Fig for the details of the activity.

https://doi.org/10.1371/journal.pone.0231165.g006

In all panels of Fig 5 where η = 0.02, the average chain length increases with μ, unless μ is too large. In Fig 5a and 5b we observe a sharp increase in the chain length for λ = {0.551, 0.601} (less pronounced for λ = {0.501, 0.651}). The sharp increase in the average chain length occurs when the bifurcation scenario changes around μ ≈ μ* (for the definition of μ* see Sec. Analysis of the dynamics and section “Dynamic bifurcation scenarios” in S1 Appendix) While middle range inhibition leads to longer sequences for ρ = 1.2, weak inhibition is more suitable for ρ = 2.4 (Fig 5c and 5d). Indeed, λ and ρ have interacting effects on chain length (***).

In Fig 6, the noise level is increased to η = 0.04. Generally speaking, increasing the noise level prolongs the chains by facilitating the activation. Especially for τ_r = 900 and λ = 0.651, the average chain length is considerably higher with η = 0.04 (Fig 6) than η = 0.02 (Fig 5). On the other hand, the relation between the average chain length and μ becomes more delicate. In Fig 6a and 6b for ρ = 1.2, the average chain length peaks for the intermediate values of μ. Strong inhibition λ = 0.651 prolongs the chains for small values of μ and slow synapses, whereas intermediate values of inhibition λ = {0.551, 0.601} favor longer chains as μ increases. For ρ = 2.4 (Fig 6c and 6d) chains are longer under weak inhibition λ = 0.501. However, increasing μ under weak inhibition considerably shortens the chains. The average chain length increases with μ under strong inhibition for λ = {0.601.0.651} Parameters λ and μ have interacting effects on chain length (***).

Our analysis unveils a nonlinear relation between μ and the chain length. When μ is small, an increase of μ provokes an increase of the length of the chain. However, in most cases we find that the chain lengths are maximal for intermediate values of μ. This is clear intuitively: large gain (small μ) prevents the units from deactivating, making the transition from one pattern to the next difficult. Small gain, on the other hand, prevents the next unit from activating. Another factor is the occurrence of the transition from scenario 1 to 2 (see Sec. Analysis of the dynamics and section “Dynamic bifurcation scenarios” in S1 Appendix). The synaptic product ρ and the global inhibition parameter λ also influence the system’s behaviour. For ρ = 1.2, inhibition in the middle range leads to longer sequences, whereas weak inhibition is more suitable for ρ = 2.4.

Analysis of the dynamics

Latching dynamics is defined as a sequence (chain) of activations of learned patterns that de-activate due to a slow process (e.g., adaptation, here synaptic depression), allowing for a transition to the next learned pattern in the sequence [11, 75]. Here we refine this description using the language of dynamics and multiple timescale analysis. The main idea is to treat the synaptic variables s_i as slowly varying parameters, so that the evolution of the system becomes a movie of the dynamical configurations of the units x_i. On the other hand the firing rate Eq (1) is well adapted to analyze latching dynamics. Indeed, from the form of Eq (1) (assuming for the moment that noise is set to 0) one can immediately see that whenever x_i is set to 0 or 1, this variable stays fixed at any time. Therefore considering any face in the hypercube [0, 1]^N defined by two coordinates (x_i, x_j), the other coordinates being fixed at 0 or 1, it is invariant under the flow of Eq (1). In other words, any trajectory starting on the face stays entirely on it. This is of course true also for the edges and vertices at the boundary of each face. Each vertex is an equilibrium of Eq (1) and connections between such equilibria can be realised through edges of the hypercube, which greatly simplifies the analysis.

When the couple (x_i, s_i) of unit i is set at (1, 1), x_i is fixed as we have seen but STD Eq (3) induces an asymptotic decrease of the synaptic variable towards the value S = (1 + ρ)⁻¹. This in turn weakens the synaptic weight J^max s_i in Eq (1), which may destabilize ξⁱ in the direction of . Considering s_i as a slowly varying parameter this can be seen as a dynamic bifurcation of an equilibrium along the edge from ξⁱ to . The following scenario was described in [40]. For the sake of simplicity we now assume i = 1 (the same arguments hold for any i). The patterns ξ¹, and ξ² lie at the vertices of a face, which we call Φ, generated by the coordinates x₁ and x₃, with x₂ = 1 and the rest of the coordinates being set to 0.

Fig 7 shows three successive snapshots of the movie on Φ. The left panel illustrates the initial configuration, with the stable pattern ξ¹ corresponding to the top left vertex. Then at some time T₀ an equilibrium bifurcates out of in the direction of ξ¹ (here the ‘slow’ STD time plays the role of bifurcation parameter, see middle panel). After a time T₁ (right panel) this bifurcated equilibrium disappears in ξ¹ which becomes unstable and a connecting trajectory is created along the edge with . Simultaneously a trajectory connects to ξ₂ along the corresponding edge. It results that the following sequence of connecting trajectories is created: . As a result, any state of the system initially close to ξ¹ will follow the ‘vertical’ edge towards , then the ‘horizontal’ edge towards ξ₂. The process can repeat itself from ξ² to ξ³ and so on. It was shown in [40] that in order to work, this scenario requires that the coefficients of the matrix J^max satisfy the relation (more generally , i = 1, …, P − 1, for the existence of a chain of P patterns), a condition which does not hold with Eq (4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Representative phase portraits of the fast dynamics on the face Φ corresponding to the scenario of [40].

The phase portraits are shown at three different ‘slow’ STD times. Green, orange and purple dots represent the stable, completely unstable and saddle equilibria, respectively. The blue lines illustrate segments of trajectory starting near ξ¹. (a) The learned patterns ξ¹ and ξ² are stable for t < T₀. (b) The bifurcation of a saddle point on the edge between ξ¹ and happens for T₀ < t < T₁. (c) Pattern ξ¹ has become unstable along the edge whilst ξ² is still stable for t > T₁. The saddle point in the interior of Φ merges with the bifurcated equilibrium before (c) is realised.

https://doi.org/10.1371/journal.pone.0231165.g007

The results of this paper rely on the observation that the existence of the connections for t > T₁ (right panel of Fig 7) is not needed for the occurrence of chains. We will show below that for the connectivity matrix J given by Eq (4) the connections exist for at most a unique value of t = T₁ and yet regular chains or segments can occur. For t > T₁ the connecting trajectory along the edge is broken by a sink (stable equilibrium) close to . In such case strong enough noise perturbations could push a trajectory out of the basin of attraction of the sink to the basin of attraction of ξ². As a result the trajectory would get past and converge towards ξ², as expected. When such chains driven by noise exist, we call them excitable chains by reference to [57] who introduced the concept. In the case when the connections exist for t = T₁ chains occur with noise of arbitrarily small amplitude, because as t approaches T₁ from above the amplitude of noise needed to jump over to the basin of attraction of ξ² converges to 0. We extend the terminology excitable chain to this case also.

Under this new scheme of excitable chains the number of possible transitions is much larger and multiple outcomes are possible. We have identified two scenarios (named 1 and 2) by which these excitable chains can occur in our problem. Typical cases are illustrated on Fig 8. As in Fig 7, snapshots of the dynamics at three different “slow” times are shown. The red line marks the boundary of the basin of attraction of ξ² and the dashed circles mark the closest distances for a possible stochastic jump out of it. In both scenarios a completely unstable equilibrium point exists on the edge from ξ¹ to the unnamed vertex on Φ, which corresponds to the pattern (1, 1, 1, 0, …, 0) (not a learned pattern).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Phase portraits of the fast dynamics on the face Φ at three different ‘slow’ STD times.

Stable patterns are coloured in green. The red trajectories are separatrices between the basins of attraction of the stable equilibria. The blue lines illustrate segments of a trajectory starting near ξ¹. Box (1) panels (a,b,c) Phase portraits illustrate the mechanisms that can lead to transition ξ¹ → ξ² with excitable connections in Scenarios 1 as time evolves. (a) Trajectory starting near ξ¹ converges to ξ¹. (b) Trajectory follows the saddle between ξ¹ and on the x₁-axis. (c) Trajectory “jumps” out of the basin of attraction of under the effect of noise and converges towards ξ² to ξ¹. Box (2) panels (d,e,f) Phase portraits illustrate the mechanisms that can lead to transition ξ¹ → ξ² with excitable connections in Scenarios 2 as time evolves. (d) Trajectory starting near ξ¹ converges to ξ¹. (e) Trajectory remains in the basin of attraction of x¹ defined by the saddle on the x₁-axis. (f) Trajectory “jumps” out of the basin of attraction of under the effect of noise and converges towards ξ².

https://doi.org/10.1371/journal.pone.0231165.g008

Under Scenario 1 the pattern ξ¹ first loses stability by a dynamic bifurcation of a sink (stable equilibrium) along the edge . The trajectory, which was initially in the basin of attraction of ξ¹, follows this sink while it is traveling along the edge towards (Fig 8b). In this time interval the distance between the sink and the attraction boundary of ξ² (the red line in Fig 8b) is decreasing. Hence it becomes more likely for the noise to carry the trajectory over the ξ² stability boundary, activating a transition to ξ². The noise level necessary for the jump becomes smaller as the sink approaches . The critical transition shown in Fig 9a occurs as the sink reaches . At this moment, noise of arbitrarily small amplitude can cause the transition. Subsequently is transiently stable and the distance to the ξ² attraction boundary increases. Therefore, it becomes likely that the trajectory remains trapped in the basin of attraction of , as shown in Fig 8c). A further decrease of s₂ occurring with the passage of time gives a loss of stability of and the trajectory leaves Φ to the inactive state. This is the mechanism of the termination of the regular part of the chain (see Fig 2 for an example).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Phase portraits corresponding to critical transitions in Scenario 1 and Scenario 2.

These phase portraits occur for a special value of σ such that changes from a saddle to a sink. (a) The phase portrait of the critical transition corresponding to Scenario 1 contains a sequence of connecting trajectories from ξ¹ to ξ², allowing for a transition from ξ¹ to ξ² with arbitrarily small noise (e.g. for ρ = 2.4, τ_r = 900, λ = 0.55, μ = 0.45, the critical transition occurs at s₁ ≈ 0.45, s₂ ≈ 0.56). (b) In the phase portrait of the critical transition corresponding to Scenario 2, a transition from ξ¹ to ξ² would fail unless the noise amplitude is sufficiently large. (e.g. for ρ = 2.4, τ_r = 900, λ = 0.55, μ = 0.15, the critical transition occurs at s₁ ≈ 0.37, s₂ ≈ 0.49).

https://doi.org/10.1371/journal.pone.0231165.g009

In Scenario 2 the overlap equilibrium point becomes a stable before ξ¹ loses stability. At the critical transition (Fig 9b), a sequence of connections from ξ¹ to ξ² does not exist, hence the trajectory cannot pass from ξ¹ to ξ² unless the noise is sufficiently large. In the context of this scenario regular chains tend to be substantially shorter.

As we have described above, noise is indispensable for crossing the attraction boundary of the next pattern, hence it is crucial for the chains we study. Minimum noise level required for jumps is scenario dependent. In Scenario 1, as s₁ and s₂ decrease along the trajectory, the distance between the sink and the ξ² attraction boundary also decreases, becoming arbitrarily small as the critical transition is approached. Thus the noise amplitude required for a jump also decreases to 0. In Scenario 2 noise needs to be stronger to make the trajectory cross over the excitability thresholds of ξ¹ and . On the other hand, we should also keep in mind that too strong noise can hinder regular chains. Simulations (and analysis, see S1 Appendix) identify μ as the main control parameter which determines the choice between these scenarios: the system follows Scenario 1 for higher values of μ and Scenario 2 for lower values of μ. This explains the difference in behavior seen in Fig 4 at lower and higher values of μ. The boundary between the two regions is defined by the value μ = μ* for which ξ¹ and change stability at the same time. For an analytic definition of μ* and more detailed analysis, see S1 Appendix. In the next section we will see how these scenarios affect irregular activation.

Irregular chains and additional numerical results

The question we address here is what happens after the last pattern of a regular segment has been reached. We let (i, i + 1) denote the last pattern of the regular sequence. It follows from previous analysis that the trajectory will remain near for a considerable amount of time. Subsequently can lose stability and the trajectory passes to the inactive state, or remains stable indefinitely. By our choice of parameters (setting I = 0, see Sec. Model) the inactive state is marginally stable, which means that an irregular activation will eventually happen due to small noise and a the next pattern will be chosen at random. The mechanism of random activation from , is similar, with the exception that chain reversal is likely to occur in this case. The transition time to an irregular activation may be significantly longer than in the case of regular transitions, which allows for the recovery of the synaptic variables. Table 1 assembles the features of regular and irregular activations, as predicted by the analysis.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Features of regular and irregular transitions as predicted by the analysis of Sec. Analysis of the dynamics.

https://doi.org/10.1371/journal.pone.0231165.t001

Our numerical results confirm the trends of the analytical predictions, at the same time showing that the dynamics of the model is more complex and other parameters play a very important role. In particular, interestingly, the longest regular segments are observed for μ values corresponding to scenario 1 close to μ = μ*, which is the boundary value separating scenarios 1 and 2 (see Sec. Analysis of the dynamics). Regular segments typically become shorter if μ is increased significantly beyond μ*, see Figs 5 and 6. This means that there exists an optimal μ window for the existence of long regular segments, or, in other words, neuronal gain needs to be not too small and not too large.

In this section we will discuss features of irregular activation, based on numerical results. To show statistics of irregular activation we define a measure of ‘distance’ Δ, as follows. Suppose at a time t, x_p and x_q are the two most recently activated units, with x_p preceding x_q in its activation. We define

Note that a regular chain satisfies Δ = 1 for all t until the last pattern is reached.

We distinguished two cases of irregular continuation of chains: reversing the chain (Δ = −1) and random reactivation of new chains (Δ ≠ −1). We allow the possibility of Δ = 1 as such chains can occur in an irregular way for the following reason: an irregular activation is typically preceded by a complete deactivation, usually with a prolonged passage time. The new activation is random, so that an activation to the pattern which is next in the overlap sequence can occur, for instance the sequence [1, 2]→[2]→[] → [3]→[2, 3]…. Such reactivations are documented in details in S3 and S4 Figs.

Recall the scenarios 1 and 2 for transitions from one pattern to the next (Fig 8). The former occurs for “large” values of μ and the latter for lower values of μ. Let be the last intermediate state at the end of the regular segment. In Scenario 2 either remains stable indefinitely or it destabilizes after some (long) time due to the repotentiation of s_p. The latter case corresponds to a dynamic scenario for a chain’s reversal. We refer to the prolonged residence of the system at as pending and note that it likely leads to a reversal. However, for high values of noise, random activation (Δ ≠ −1) may also occur. The scenario 1 is more likely to yield random re-activation as loses stability in the x_p+1 direction with the decrease of s_p+1, so that a transition to the inactive state is possible. Notice that in this case too other Δ values are possible when the noise is large. Statistics on distance of irregular chains show main effects of parameter η (*), μ (***), λ (*) and ρ (**). The four parameters η (*), μ (***), λ (*) and τ also have effects on activation distance and interact with each other (***).

Figs 10 and 11 show the average activation distance Δ for η = 0.02 and η = 0.04, respectively. For λ = {0.501, 0.551} and small values of μ, the system remains on the last activated pattern. Activity with Δ = −1 is generally supported for ρ = 1.2 and for ρ = 2.4 if (μ, λ) are small. Indeed, λ, ρ and μ have interacting effects on the activation distance (***). We do not see any activation for small values of μ in Fig 10 which indicates that the activity remains either on a pattern ξ or on an intermediate state . Increasing μ introduces a backwards activation, except for λ = 0.651 for which the new activity is in the forward direction. For ρ = 2.4, we observe that the average distance increases with λ and μ if τ_r = 300, but decreases with μ if τ_r = 900.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Probability of a new activity and average distance Δ for noise η = 0.02.

Synaptic time constant equals to τ_r = 300 in panels (b) and (e), and τ_r = 900 in panels (c) and (f). (a, b, c) Activity for ρ = 1.2. (a) The probability of a new activity after the initial sequence for ρ = 300 (upper panel) and τ = 900 (lower panel). Small values of μ (1/neural gain) tends to keep the system on the last activated pattern. Minimum μ value required for a new sequence decreases with inhibition λ and τ_r. (b, c) Activated patterns mostly remain in negative distances except for λ = 0.651 for τ_r = {300, 900}, for high values for μ for λ = 0.601 with τ_r = 300. (d, e, f) Activity for ρ = 2.4. (d) The probability of a new activity after the initial sequence for ρ = 300 (upper panel) and τ = 900 (lower panel). Minimum μ value required for a new sequence decreases with inhibition λ and time constant τ_r. A new activation is always observed for τ_r = 300, λ = 0.651; and for τ_r = 900, λ = {0.601, 0.651}. (e) Average distance Δ is positive and increases with μ. (f) Average distance Δ is negative and increases with mu for λ = 0.501. Average distance Δ is positive and decreases with μ for λ = {0.551, 0.601, 0.651}.

https://doi.org/10.1371/journal.pone.0231165.g010

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Probability of a new activity and average distance Δ for noise η = 0.04.

Synaptic time constant equals to τ_r = 300 in panels (b) and (e), and τ_r = 900 in panels (c) and (f). (a, b, c) Activity for ρ = 1.2. (a) The probability of a new activity after the initial sequence for ρ = 300 (upper panel) and τ = 900 (lower panel). Minimum μ (1/neural gain) value required for a new sequence decreases with inhibition λ and time constant τ_r. (a, b) Average distance is negative except for μ > 0.3, λ = 0.651 and τ_r = 300. (d, e, f) Activity for ρ = 2.4. (d) The probability of a new activity after the initial sequence for ρ = 300 (upper panel) and τ = 900 (lower panel). A new activation is always observed for τ_r = 300, λ = {0.601, 0.651}; and in almost all trials for τ_r = 900.(e) Average distance Δ is positive except for λ = 0.501 where it increases from negative to positive with increasing μ. (d) Average distance Δ decreases for all cases except for λ = 0.501 where it increases from negative to positive with increasing μ.

https://doi.org/10.1371/journal.pone.0231165.g011

Increasing the noise level η (Fig 11) facilitates activation of new patterns. As Fig 11a demonstrates, the system remains on the last activated pattern only for λ = {0.501, 0.551} and small values of μ, New activation mostly stays in the negative distance for ρ = 1.2 unless for high values λ and μ (Fig 11b and 11c). Indeed, there are interactive effects between λ, η and ρ (**) and between λ, ρ and μ (**). Taking ρ = 2.4 considerable changes the average Δ for both synaptic time constants. Probability of a new activity is above 0.5 for all parameter combinations (Fig 11d). For τ_r = 300, the average stays in the positive region for the whole range of μ, unless λ = 0.501 for which average Δ climbs from negative to positive values (Fig 11e). A similar pattern is observed with λ = 0.501 and the synaptic time constant τ_r = 900 (Fig 11f). However, the average Δ decreases with μ and for the other values of λ when τ_r = 900.

Supporting materials S3 and S4 Figs show the percentage of Δ values after a new activation for η = 0.02 and η = 0.04, respectively. Recall that as the chains get longer with increasing μ, regular segments get longer, as well, specially when noise is high (η = 0.04). Regarding the type of forward and/or backward irregular chains, high values of ρ and high values of μ (e.g. low gain) for low values of noise η (S3 Fig) increase the possibility for irregular chains in the forward direction, while the combination of high values of μ (e.g. low gain), ρ and noise η increase the possibility for irregular chains in both directions (S4 Fig). The difference between the percentages of Δ for τ_r = 300 and τ_r = 900 indicates the capability of slow synapses to yield longer chains.

In order to see the relation between the chain length and distance Δ for each combination of (η, ρ, τ_r), we categorized the regular sequences with respect to the last activated pattern. Then, we extracted the Δ values among the trials with activation and obtain a Δ set for a each possible last activated pattern of the sequence ABCDEF. Fig 12 shows the distribution of Δ at patterns from A to F in a violin plot. Left side of each violin shows the results with η = 0.02, and right side with η = 0.04. We remark at first glance that Δ depends on the chain length. Activation is in the forward direction in short chains whereas it is in the backward direction for long chains (still preserving a preference for Δ = −1). While this is related to being a bounded system (in terms of the system size N = 8), for the intermediate patterns, like D, negative and positive values of Δ are almost equally distributed (especially for ρ = 2.4, τ_r = 900). We also observe that after passing C, the system activates more and more the units in negative distance Δ ≠ −1. Increasing noise spreads the distribution of Δ, very visibly in ρ = 1.2 for patterns A, B; also for pattern F in ρ = 2.4, τ_r = 900. Finally, distribution of Δ approximates to a normal distribution for shorter chains.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Distribution of activation distance Δ at last activated patterns from A to F over all trials.

The distributions are bounded by the minima and maxima of the Δ sets. Each violin is colored with respect to the color code of the last activated units of a pattern in the forward direction as in Figs 1 and 2. Left half of each violin corresponds to activity with noise η = 0.02 and the hashed right halves correspond to activity with noise η = 0.04. (a) Activity for ρ = 1.2, τ_r = 300. (b) Activity for ρ = 1.2, τ_r = 900. (a) Activity for ρ = 2.4, τ_r = 300. (b) Activity for ρ = 2.4, τ_r = 900.

https://doi.org/10.1371/journal.pone.0231165.g012

Perturbed connectivity matrix

In order to test the robustness of our model, we randomly perturbed the off-diagonal elements of the connectivity matrix (4) while ensuring its diagonally symmetric structure. We consider two levels of perturbations (5% and 10%) and a parameter set for which we obtain a wide range of behaviour with matrix (4) (the parameter set: η = 0.04, ρ = 1.2, τ_r = {300, 900}). The simulation results are presented in Table 2 and in S5 Fig. Looking at Table 2, we see that the difference in the chain lengths are less than 10%, and in the probability of a new activity is around 15%. Furthermore, these two features follow similar patters to the ones of the regular matrix as S5(a)–S5(d) Fig show. The differences between the average distance Δ values are around 18% for τ_r = 300 and 32% for τ_r = 900. In particular, the difference in Δ increases with μ and λ for τ_r = 900 S5(e) and S5(f).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Relative difference between the features of the unperturbed and irregularly perturbed synaptic matrices.

https://doi.org/10.1371/journal.pone.0231165.t002

Synaptic matrix

In the present model the overlap had the same number of shared units for all the overlapping populations. This allowed us to show that variable overlap is not a necessary condition for the activation of sequences of populations. A consequence of the constant overlap is that sequences from a stimulus-driven end-point pattern in the sequence (e. g. first pattern A of the sequence) are directional but sequences from a mid-point pattern can go in any of the two possible directions. The model can then generate bi-directional sequences interesting in free recall. Starting from the first pattern A (or G), the sequence ABCDEFG is oriented in one direction (or in the other direction), and starting from a middle pattern e.g. D, the sequence can be oriented in any of the two possible directions. The present model allows for bi-directionnal sequences as well as for new sequences depending on the value of neuronal gain γ = μ⁻¹.

Our model is robust against small perturbations of the connectivity matrix. In other words our results would hold, with small modifications, in a slightly heterogeneous network.

Regular vs. irregular sequences

Regarding regular sequences, the chain length increases with noise and for combinations of strong STD (high values of ρ) and low inhibition, or weak STD (low values of ρ) and strong inhibition. Further, for most combinations of noise, STD and inhibition, there is an optimal value of gain that generates the longest chains. The sensitivity of a neuron to its incoming activation varies with changes in its gain [65]. Simulations and analysis show that the neuronal gain (1/μ) is a key control parameter that selects the length and type of sequence activated: regular or irregular. Large neuronal gain impairs the deactivation of the units in a pattern and hence makes the transition to the next pattern difficult, and small gain impairs the activation of the next unit and again makes the transition difficult. Consequently there is an optimal window for the gain corresponding to long sequences. Experimental evidence shows that presentations of a given stimulus reproduces the same sequence reliably [14, 16, 22, 77]. The present model can repeat systematic full sequences of activation for some values of the parameters that make the network change patterns in a given order. This ‘reliable’ mode could be well adapted to the reliable reproduction of learned sequences of behaviors.

Regarding irregular sequences, a large neuronal gain leads to the second scenario describing transitions from one pattern to the next (as in Fig 8). According to this second scenario, the last intermediate state of the network at the end of a regular segment can destabilize and leads to a reversal of the sequence. In that case the network activates patterns backward in the reverse order. Further, for high values of noise, Scenario 2 can lead to random activation of patterns in either the forward or backward direction. Such variable sequences are more likely to be generated according to the first scenario that makes possible a transition to another state in the forward or backward direction and that does not necessarily overlap with the current state (forward or backward leaps). Direction of recall has been linked to the stimulation amplitude presence of non-context units in [78]. Our model can generate variable sequences over repetitions of the same triggering stimulus for high values of gain, in line with a memoryless system [56] that activates a new pattern in an unpredictable fashion. Behavioral studies indicate that presentation of a triggering stimulus can activate distant items that are not directly associated to it [79]. The generation of new sequences corresponding to the activation of new possibilities [80] and the execution of new information-seeking behaviors such as saccades or locomotor explorations of unknown locations [1] rely on variable internal neural dynamics such as in the medial frontal cortex [81]. This ‘creative’ mode of variable activation not following a given sequence could correspond to a mind wandering mode [19, 82] or divergent thinking involved in creativity [83–86].

Neuromodulation of the switch between regular and irregular sequences

Neuronal gain is reported to depend on neuromodulatory factors such as dopamine [51, 87–89] involved in reward-seeking behaviors and punishment [90–92]. Dopamine is reported to modulate the magnitude of the activation between associates in memory (priming; [93, 94]) and dopamine induced changes in neuronal gain have been reported to account for changes in activation in memory [52] and for changes in neuronal activity that controls muscle outputs [95]. A novel feature of our network model is that neuronal gain influences the type of sequences that are generated: regular or irregular. Typical computational models of sequence generation reproduce learned sequences [15]. However, if the brain must in some case reproduce systematic behaviors, it must also have the capacity to liberate itself from repetition in order to create new behaviors. The present research shows that the network can exhibit the dual behavior of activating regular or irregular sequences for a given synaptic matrix. The transition depends on biological parameters, in particular on gain modulation. Given that changes in gain change the length of the regular sequence, and that when the regular sequence stops it becomes irregular, the gain controls the regularity of the sequences. The present research sheds light on how the brain can switch between a ‘reliable’ mode and a ‘creative’ mode of sequential behavior depending on external factors such as reward that neuromodulate neuronal gain.

Fixed versus increasing overlap size

Earlier works [40–42] proved the existence of regular sequences under the assumptions of increasing overlap and synaptic efficacy. In this work we showed that neither of these conditions is needed: regular sequences can exist in the context of equal overlap. It is know that populations of neurons coding for memories, and their overlaps, can vary (due to learning) on time scales that are long in the context of this paper, but still relatively short [96]. Sequences arising through increasing overlap can be understood as learned as opposed to the regular sequences in this paper, that occur merely due to the semantic relation between concepts. Consequently, it is interesting to extend our modelling framework to a setting where overlap could vary on a super-slow timescale.