Ensemble-based neural sampling

In neural sampling [4], each neuron in a network represents a binary random variable. Spike generation is stochastic, with a probability that depends on the current membrane potential. At each time , the activity of each neuron is mapped to a sample for the value of the corresponding variable by setting the value to 1 if the neuron has spiked in for some small (e.g. 20 ms). It was shown that under certain conditions on the membrane potentials of the neurons, the network converges to a stationary distribution that corresponds to the posterior distribution of the represented random variables for the given evidence. Evidence is provided to the circuit by clamping the activities of a subset of neurons during inference. The marginal distribution for a given variable can be read out by observing the firing activity of the corresponding neuron in the stationary distribution.

Neural sampling is an implementation of the Markov Chain Monte Carlo (MCMC) sampling approach (see e.g. [6]) in networks of spiking neurons. By definition, it does not provide a suitable model for the representation of time-varying distributions, since samples are generated in a sequential manner. Convergence to the stationary distribution in MCMC sampling can take substantial time, the readout of marginal distributions demands spike counts of neurons over extended periods of time, and MCMC sampling is only defined for the fictional case of stationary external inputs. But also time-varying distributions can theoretically be handled through Monte Carlo sampling if one has a sufficiently parallelized stochastic system that can generate at each time point simultaneously several samples from the time varying distribution , and by carrying out simple computational operations on this batch of samples in parallel. The resulting computational model is usually referred to as particle filter, a special case of sequential Monte Carlo sampling [6], [7]. Here each sample from a batch that is generated at time is referred to as a particle.

To port the idea of neural sampling to the representation of time-varying distributions through neuronal ensembles, we therefore consider ensembles that collectively encode the belief about the value of a random variable with range in terms of a probability distribution . We refer to the value of a random variable also as the hidden state, or simply the state of the variable. We will in the following omit the subscript in for notational convenience (formally, we consider a family of variables, indexed by , that defines a random process, see [8]; is then the distribution over the member of this family). Each ensemble consists of neurons , where we refer to as the ensemble size. We interpret a spike in the circuit as one sample from , i.e., one concrete value for the represented variable drawn according to the distribution [4], [9]. In particular, if a spike is elicited by some neuron in ensemble , then this value is . See Figure 2A for an illustration of sample-based representations.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Spikes as samples from probability distributions.

A) Sample-based representations of probability distributions. True distribution of a random variable (green) and approximated distribution (yellow) based on 20 samples (top) and 200 samples (bottom) B) Interpretation of the spiking activity of two neuronal ensembles as samples from a probability distribution over a temporally changing random variable . Shown is an example for a random variable with two possible states. Black lines in the top traces indicate action potentials in two ensembles (5 neurons per state shown). Traces above spikes show EPSP-filtered versions of these spikes (red: state 1; blue: state 2). Bottom plot: Estimated probabilities for state 1 (red) and state 2 (blue) according to eq. (1) based on the spiking activity of 10 neurons per state.

https://doi.org/10.1371/journal.pcbi.1003859.g002

A downstream neuron can evaluate how many spikes it received from each ensemble within its membrane time constant through summation of excitatory postsynaptic potentials (EPSPs) caused by spikes from ensemble neurons. We denote the EPSP-filtered spike train of neuron by (see Methods for a precise definition) and adopt rectangular EPSP shapes of length in the following, similar to those recorded at the soma of pyramidal neurons for dendrite-targeting synaptic inputs (see Figure 1 in [10]). The sum of all EPSPs from an ensemble , denoted by , is then the number of samples for hidden state in the time window from to . The number of samples is thus directly accessible to downstream neurons. We refer to as the (non-normalized) probability mass for hidden state . The use of plateau-like EPSP shapes is motivated from the need to count spikes in some predefined time interval. An alternative motivation that is based on the idea that the EPSP weights a spike by the probability that it belongs to the most recent samples is given in Text S1.

The represented distribution can be estimated by the relative portion of spikes from the ensembles(1)where we assume that at least one sample is available. See Figure 2B for an intuitive illustration of ENS coding. In this representation, probabilities are temporally filtered by the EPSPs. Hence, the ability of the code to capture fast dynamics of distributions depends on the length of EPSPs, where shorter time constants give rise to faster tracking. Due to the stochasticity of the sampling process, is a random variable that assumes different values each time the distribution is represented. We demand in ENS coding that the expected value of is equal to the temporally filtered represented probability for all states , see Methods for details.

We will see in the construction of computational operations in ENS coding that downstream neurons do not have to carry out the division of eq. (1). Instead, for these operations, they can compute with the non-normalized probability masses that they obtain through summation of EPSPs from ensemble neurons. The reason is that normalization is not necessary in the representation of a distribution in ENS coding. It is rather the relative portion of spikes for each value of the random variable that defines the represented distribution. Of course, activity needs to be kept in some reasonable range, but this can be done in a rather relaxed manner.

Basic properties of the ENS code

Assume that samples (spikes) for state are produced by a Poisson process with rate proportional to the represented probability, i.e., the rates of neurons in ensemble are given by , where is a constant that defines the maximal instantaneous rate of each neuron. is then a random variable distributed according to a Poisson distribution with intensity , where the estimation sample size is the average number of spikes produced by all ensembles within a time span of .

We show in Methods that in this case, is an unbiased estimator of the probability of state at time filtered by the EPSPs, hence, an ENS code is established. An important question is how parameters of a circuit influence the fidelity of the encoding. To answer this question, we investigated the variance of the estimator. It is inversely proportional to the ensemble size , the maximal firing rate , and the membrane time constant if the number of samples within is large, see Methods. Hence, the accuracy can be increased by increasing the ensemble size, the firing rate of neurons, and the time constant of neuronal integration. Note however that an increase of the latter will lead to more temporal filtering of the distribution.

We close this discussion by considering the relation between the instantaneous firing rates of ensemble neurons and the mean represented probability mass . The probability mass is not an instantaneous function of the ensemble firing rate, since at time there are still past samples that influence through their EPSPs. A past sample becomes invalid after time , when the associated EPSP vanishes. The instantaneous firing rates change the mass through the production of novel samples. Consider given continuous firing rates . Under mild assumptions on the firing rates of ensemble neurons (see Methods), the change of the expected probability mass is then given by(2)where the expectation is taken over realizations of spike trains for the given instantaneous rates. Here, the first term is due to the production of novel samples and the second term due to old samples that become invalid. In summary, the membrane potentials of the neurons determine – through the firing rate – the rate of change of the represented probability mass in ENS-coding.

Computational operations through ensemble-based neural sampling

We address now the question how basic computations on time-varying internal beliefs can be realized by neural circuits in ENS coding. Spiking activity of excitatory neurons is modeled according to the stochastic Spike Response model [11], [12]. In this model, each neuron emits a Poisson spike train with instantaneous firing rate(3)

Here, denotes a link function that links the somatic membrane potential to the instantaneous firing rate. Typically, the link function is either an exponential function or a non-negative linear function. We consider in this article a non-negative linear link function [12], where is for non-negative and otherwise.

We discuss five classes of computational tasks.

Task class A: Simple probabilistic dependencies

We first discuss how the belief for a random variable can be inferred in ENS coding given the belief over a variable . Consider a random variable for which the distribution depends solely on the current state of a random variable . The task is to infer the distribution over for the current distribution over . This operation is needed for example in typical decision making tasks where inference about a rewarded action has to be performed according to the belief about some hidden variable that is based on sensory information. For example, the rewarded movement direction has to be guessed, based on the belief about the perceived cue combination in the ambiguous target task.

Assume that is constant and represented by neurons through ENS coding with estimation sample size . We are looking for a neural circuit that represents the posterior belief(4)in ENS coding. Here, are the known conditional probabilities that determine the dependencies between and .

We consider a layer of ensembles that receive feed-forward synaptic input from ensembles representing , see Figure 3. The membrane potentials of neurons are given by(5)where denotes the efficacy of the synapse connecting presynaptic neuron to postsynaptic neuron (we assume for simplicity of notation that all weights between two ensembles are identical). Consider the estimator for synaptic efficacies(6)where is the ensemble size of the ensembles , and is some constant. Due to the stochastic nature of neurons, this estimator is a random variable. Its expected value (with respect to realizations of spikes trains in all ensembles) is equal to the posterior probability and the estimation sample size is (see Methods). Thus, the layer represents the posterior distribution in ENS coding. The estimate at some specific time is however variable due to variability in spike counts of both the representation of and the representation of . An analysis of the variance of the posterior representation is given in Methods.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Computations in ENS coding in a feed forward circuit architecture.

A binary random variable is represented in ENS coding through neurons . The posterior for a binary variable is represented by neurons . Each variable is represented by ensembles, one for each possible state (indicated by neuron color), and neurons per ensemble. The two layers are connected in an all-to-all manner. Arrows indicate efferent connections (i.e., outputs in ENS coding). The architecture is summarized in the inset.

https://doi.org/10.1371/journal.pcbi.1003859.g003

We will use such a layer with feed-forward input several times in our simulations to infer a belief about rewarded actions for a given belief about the state of a random variable and thus refer to it as an action readout layer.

We will also need a special case of this operation where the distribution is simply copied, i.e., the state of the random variable is assumed to be identical to the state of . In other words, is 1 for and 0 for . The copy operation is thus performed for weights and for (where we used ).

Task class B: Evidence integration

In pure evidence integration, the value of the random variable is assumed to be constant and only indirectly observable via stochastic point-event observations. Point-event observations are assumed to arise according to Poisson processes with instantaneous rates that depend on the current hidden state. In the context of neuronal circuits, observations are reported through spikes of afferent neurons . In particular, afferent neuron is assumed to spike in a Poissonian manner with rate if the hidden variable assumes state . For a prior distribution over states , the task is to infer the posterior , that is, the distribution over states at time , given the spike trains of all afferent neurons up to time , denoted here by . Many laboratory tasks can be formalized as evidence integration task, including the random-dot motion task, the probabilistic inference task considered in [2] and the ambiguous target task discussed in the introduction. We construct in the following a circuit of spiking neurons that approximates optimal evidence integration by performing particle filtering in ENS coding under the assumption that is constant. The will then evaluate its behavior against experimental data in computer simulations. This circuit will be extended in the subsequent sections to perform particle filtering for tasks classes C and D.

It is well-known that the evidence integration problem can be solved efficiently through a set of coupled differential equations(7)where is the spike train of afferent neuron formalized as a sum of Dirac delta pulses at spike times (see Methods), and [13], [14]. The inferred probabilities can be obtained by normalization (for ). induces a constant decrease of such that hidden states that give rise to many observations are punished if no observations are encountered. Is the dynamics (7) compatible with ENS coding, assuming that is estimated from the spiking activity of an ensemble? Four potential difficulties arise. First, the afferent neurons impact eq. (7) via point events and not via EPSPs ( instead of ). Second, the deterministic dynamics (7) need to be implemented via particles in the ENS code. Third, the summed evidence needs to be multiplied with the current value of . And finally, to avoid an exponential blow-up of the 's, their values need to be normalized. We discuss these four issues in the following.

Particle-based implementation of the filtering equations.

We now discuss how eq. (8) is approximated in ENS coding. While eq. (8) is a deterministic differential equation, the dynamics of the circuit is stochastic due to sampling noise. We construct a circuit that approximates the desired changes in its expected probability masses. The validity of this approach will later be ascertained through various computer simulations. The belief about the hidden state of the random variable is in general shaped by two components. First, the assumed internal dynamics of the random variable, and second by novel evidence about the state. Consider a circuit that consists of two layers (the dynamics layer) and (the evidence layer) with neural ensembles and respectively, see Figure 4A. The dynamics layer implements changes of the represented distribution due to the internal dynamics of the random variable. The evidence layer implements changes due to incoming evidence. Since in task class B, the random variable is assumed to be static, no temporal changes of the random variable are expected, and hence the excitatory weights from to are set such that copies the distribution represented by , as discussed in Task class A. For task classes C and D, different weights will be used such that layer predicts dynamics changes of the hidden variable.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Particle filter circuit architecture for task classes B and C.

A) Circuit with ensembles (indicated by red and blue neurons respectively) and neurons per ensemble. Neurons in layer receive synaptic connections from neurons in layer and update the represented distribution according to evidence input from afferent neurons (green). Lateral inhibition (magenta; see panel C) stabilizes activity in this layer. Neurons project back to layer . For task class B (evidence integration; static random variable ), only connections between neurons that code for the same hidden state are necessary and layer simply copies the distribution represented by layer , see Task class A and Figure 3 (in contrast to Figure 3, the copying ensembles are plotted above ensembles in order to avoid a cluttered diagram). For task class C (Bayesian filtering; random variable with time-independent dynamics), implements changes of the represented distribution due to the dynamics of the random variable and is potentially fully connected to . Neurons in layer disinhibit neurons in layer (double-dot connections; see panel B). Disinhbition and lateral inhibition is indicated by shortcuts as defined in B, C. Arrows indicate efferent connections. A schematic overview of the circuit is shown in the inset. B) Disinhibition : neurons excite an interneuron (purple) which inhibits the inhibitory drive to some neuron . As a graphical shortcut, we draw such disinhibitory influence as a connection with two circles (inset) C) Lateral inhibition: Pyramidal cells (blue) excite a pool of inhibitory neurons (magenta) which feed back common inhibition . The graphical shortcut for lateral inhibition is shown in the inset.

https://doi.org/10.1371/journal.pcbi.1003859.g004

The evidence layer receives input from and evidence from afferent neurons . Our goal is that its probability masses integrate evidence in the representation of in , such that(9)where the latter approximation applies due to the copy operation of layer . This equation resembles eq. (8) where the 's represent the 's. We show in Methods that such changes are obtained if the membrane potentials of the neurons in are set to(10)where are positive biases.

Stabilization of firing rates through lateral inhibition.

Lateral inhibition is generally assumed to stabilize the activity of excitatory populations [21]–[23]. A group of pyramidal cells inhibit each other laterally by projecting to a group of inhibitory neurons which in turn inhibit that ensemble, see Figure 4C. The key observation that enables us to use lateral inhibition to stabilize circuit activity is that one has freedom to choose in eq. (12) as long as it is identical in all ensembles. We model lateral inhibition that depends on the recent firing activity in such that inhibitory activity increases if the estimation sample size is above the desired value and choose , see Methods for details and a brief discussion. This concludes the construction of a particle filtering circuit in ENS coding for task class B. The circuit architecture is depicted in Figure 4A. A summary of circuit equations can be found in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Particle filter circuit equations for task classes B and C.

https://doi.org/10.1371/journal.pcbi.1003859.t001

We tested how well a circuit consisting of 2000 neurons per state and an estimation sample size of 400 can approximate the true posterior in a simple evidence integration setup. The task was to compute the posterior distribution for a random variable with two hidden states and two observable variables , see Figure 5A. The schematic circuit diagram is shown in panel B. The two evidence neurons spiked at times 20 ms and 25 ms respectively, see panel C. Figure 5D depicts the rate dynamics in layer for one example trial. Novel evidence transiently increases the firing rate in the layer, which is in turn restored by lateral inhibition. The response of ensemble that represents state 1 undergoes a transient increase that is counteracted by inhibition until it stabilizes at an enhanced sustained level. This behavior is reminiscent of the typical response of cortical pyramidal cells to sensory input. Figure 5E shows the temporal evolution of the encoded posterior probability in comparison to the true posterior . The true posterior is approximated very well after a delay of about 20 ms, which is the time needed to integrate the EPSPs from evidence neurons. We simulated 100 trials where in each trial, prior probabilities for the states and observation likelihoods were drawn randomly such that the posterior at time t = 45 ms assumed values between 0 and 1 (see Methods). The estimate of the circuit at the end of the second EPSP (i.e., at time t = 45 ms) is shown in comparison to the true posterior in Figure 5F.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 5. Evidence integration through particle filtering in ENS coding.

A) The state of a binary random variable that gives rise to two possible observations is estimated. Both observations occur more frequently in state 1 (indicated by sharpness of arrows). B) Estimation is performed by a particle filtering circuit with evidence input (: dynamics layer ensembles; : evidence layer ensembles). C) An evidence spike is observed at times 20 ms and 25 ms in evidence neuron and respectively. D) Example for the rate dynamics in layer . Ensemble rate for ensemble (black) and whole layer (gray). The input leads to a transient increase in the ensemble rate. Inhibition recovers baseline activity. The ensemble rate for state 1 undergoes a transient and a sustained activity increase. E) Temporal evolution of estimated posterior probability for state 1 (black) in comparison to true posterior (gray) for this example run. F) Posterior probability at t = 45 ms () for state 1 of the circuit in comparison to true posterior at this time (). Each dot represents one out of 100 runs with prior probabilities and observation likelihoods drawn independently in each run (see Methods). The results of the example run from panels A–E is indicated by a cross.

https://doi.org/10.1371/journal.pcbi.1003859.g005

Gating of activity for multiplication can be used for many types of multiplicative operations on probability distributions. In Text S2, we discuss its application to cue combination, an operation that has been considered for example in [24].

Comparison to experimental results

We performed computer simulations in order to compare the behavior of the model to various experimental studies on tasks that are examples for task class B.

The ambiguous target task.

The ambiguous target task studied in [3] was already discussed above, see also Figure 1A. In our model of the decision making process, the hidden state of a random variable was estimated through evidence integration. Each of the 16 hidden states corresponded to a tuple , where denotes one of eight possible directions of movement, and denotes the color of the color cue. In other words, such a state represented the color of the color cue and the movement direction that leads to reward, see Figure 6Aa. Possible observations were the fixation cross, the spatial cues at 8 positions in two colors, and the two color cues. Each of the 19 possible stimuli was coded by 20 afferent neuron that fired at a baseline rate of 0.1 Hz. When a stimulus was present, the corresponding neurons spiked in a Poissonian manner with a rate of 5 Hz.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 6. Particle filtering in ENS coding for the ambiguous target task.

A) Represented random variables. Aa) Evidence integration is performed for a random variable with 16 hidden states corresponding to direction-color pairs. Values of the random variable are depicted as circles. Observations accessible to the monkey in one example state are shown as boxes. Ab) The action readout layer infers a color-independent random variable by marginalization over color in each direction. B) Circuit structure. The circuit on the top approximates evidence integration through particle filtering (top gray box; : dynamics layer ensembles; : evidence layer ensembles)) on the random variable indicated in panel (Aa). An action readout layer (bottom gray box; ensembles X) receives feed-forward projections from the particle filter circuit. C) Spike rasters from simulations for afferent neurons (Ca) and neurons in the action readout layer (Cb). Each line corresponds to the output of one neuron. Afferent neurons are ordered by feature selectivity (e.g., top neurons code the presence of the fixation cross). Action readout neurons are ordered by preferred movement direction. See also Figure 1.

https://doi.org/10.1371/journal.pcbi.1003859.g006

We simulated a particle filter circuit to compute the belief about the state of the random variable with 1000 neurons per hidden state and an estimation sample size of 400. An action readout layer as described in Task class A was added that received connections from in a feedforward manner, see Figure 6B. This layer computed the current belief over rewarded actions independently from the color of the color cue (i.e., it marginalized over color), see Methods for details.

The spiking activity of afferent neurons that provided evidence for one example simulation run is shown in Figure 6Ca. Simulated neural activities from the readout layer are shown in Figure 6Cb, see also Figure 1C. After the spatial cue was presented, the two consistent ensembles increased their activity. Due to competition between these ensembles, neurons fired at a medium rate. After the color cue was shown, only the ensemble consistent with both the spatial and the color cue remained active. These neurons increased their firing rate since the competing action became improbable and the winning ensemble was uncompeted. This behavior has been observed in PMd [3], see Figure 1B. The action readout layer is not needed to reproduce this behavior, since neurons in the particle filter circuit exhibit similar behavior. However, it was reported that most neurons in PMd were not color selective [3]. In our model, neurons of the particle filter circuit are color selective since states are defined according to direction-color pairs. It is clear that color-related information has to be integrated with movement-related information and memorized during the memory epoch in order to solve the task. The experimental results suggest that this integration is not implemented in PMd but rather in upstream circuits. PMd could then act as a motor readout.

Applications of the model to various other experimental tasks can be found in supporting texts. Action-predictive activity in macaque motor cortex is also modulated by the expected value of the action. This was demonstrated for example in [25]. An application of our model to this scenario is described in Text S3. Furthermore, we show in Text S4 that the model is consistent with features of neuronal activity during random-dot motion tasks [1], [26], [27]. In Text S5 it is shown that the model can also explain neuronal activitiy in area LIP during a probabilistic reasoning task [2].

Task class C: Bayesian filtering

Evidence integration cannot take temporal changes of the hidden variable into account. Knowledge about temporal changes can be exploited by Bayesian filtering, which is an extension of evidence integration. Here, we assume that the dynamics of are constant during the filtering process. The more general case when the dynamics may change is discussed below in Task class D. Formally, the Bayesian filtering problem considered here is to estimate the posterior distribution over the states of a random variable that represents the hidden state of a random process which is only indirectly observable via stochastic point-event observations . In particular, state changes are assumed to be Markovian with transition rates for each pair of distinct states . Transition rate defines the rate of transition from state to state , i.e., the probability that a transition occurs to state in some small time interval if the current state is , thus defining a continuous time Markov chain, see Methods for a more formal description.

Note that evidence integration is a special case of Bayesian filtering with the assumption of no state transitions, i.e., for all . In the following, we show that the particle filtering circuit for task class B constructed above can easily be extended to this generalization. The Bayesian filtering problem can be solved efficiently through a set of coupled differential equations(13)where again the inferred probabilities are obtained by normalization [13], [14]. Note that the first term in eq. (13) is identical to eq. (7), since the optimal solution for evidence integration, eq. (7), is the special case of eq. (13) for vanishing transition rates. This term is taken care of in layer of the particle filtering circuit for task class B. In this circuit, simply copies the distribution given by . We modify the connections from to such that instead provides the changes in probability masses needed for the second and third term, i.e., it predicts changes based on the assumed dynamics of the random variable. approximates the desired changes of probability masses if the membrane potentials are given by(14)with synaptic efficacies for and for all , see Methods. An overview of the circuit equations and parameters is given in Table 1, see also Figure 4A.

We tested the ability of a particle filter circuit consisting of 2000 neurons per state and an estimation sample size of 400 to track the temporal evolution of a binary random variable where state 1 transitions to state 2 with some transition rate , see Figure 7A. The dynamics of the circuit and the estimated probability for an example simulation run are shown in panels C,D. We simulated 100 trials, where in each trial the transition rate was drawn uniformly in [0, 30]Hz and initial probabilities were drawn uniformly in . The estimate of the posterior at time t = 50 ms is shown in comparison to the true posterior in Figure 7E. See Methods for details on this simulation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 7. Tracking of dynamics in ENS coding.

A) The state of a binary random variable is estimated where state 1 transitions to state 2 with some transition rate . B) Estimation is performed by a particle filter circuit without evidence input (: dynamics layer ensembles; : evidence layer ensembles). C) Example for the rate dynamics in layer . Ensemble rate for ensembles (black) and whole layer (gray). While rates in ensembles change due to the prediction of a transition, inhibition keeps the overall firing rate in the layer approximately constant. D) Temporal evolution of estimated posterior probability for state 2 (black) and true posterior (gray) for this example run. E) Circuit estimates of posterior probabilities at time t = 50 ms () in comparison to true posteriors at this time (). Shown are 100 runs (dots) with prior probability for state 1 and transition rate drawn from uniform distributions in [0.1, 0.9] and [0, 30]Hz respectively in each run. The result of the example run from panels C-D is indicated by a cross.

https://doi.org/10.1371/journal.pcbi.1003859.g007

Particle filtering in a generic setup for task class C.

We performed further computer simulations in order to test the performance of the model in a generic setup. In this setup, a random variable evolved according to a continuous time Markov chain with five states. In this Markov chain, state 1 transitions to states 2 or 3 which themselves transition to states 4 and 5 respectively. From states 4 and 5, a transition to state 1 is possible, see Figure 8Aa (bottom). Transition rates for all possible transitions were set to 1Hz. Information about the actual state was conveyed by 35 afferent neurons with state-dependent rates defined by Gaussians as shown in Figure 8Aa (top). Note that states 2 and 3 gave rise to very similar observations. Thus, many observations have to be integrated before these states can be distinguished. This makes inference over the current state hard if the full distribution over state probabilities is not communicated over time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 8. Particle filtering for task class C (A) and task class D (B) in the ENS code.

Aa) State diagram of the Markov chain for the dynamics of the hidden random variable (bottom) and state-dependent firing rates of afferent neurons (top). Colors indicate the value of the hidden state. Ab) Actual hidden state over time is indicated by color in correspondence with colors in panel Aa. Ac) Spike trains of afferent neurons. Each line corresponds to the output of one afferent neuron ordered according to panel Aa. Ad) Network response to the input in panel Ac. Neurons are ordered according to their preferred state from state 1 (top neurons) to 5 (bottom neurons). Ae) Network belief (estimated posterior state probability) derived from network activity. Rows ordered by state as neurons in panel Ad. Hot color indicates high probability of the state. Note the uncertainty when state 2 or 3 is entered. Af) Summary of network performance (“model”; fraction of incorrect state estimates) in comparison with the optimal Bayesian filter (“opt”), a network with jittered synaptic efficacies (“jit”), and the optimal decision based on the most recent observation only (“inp”). Bars are means and errorbars STDs over 20 state and observation sequences (12 seconds each). B) Particle filtering for task class D. Ba) As panel Aa but with context. Dark gray arrows in the state diagram indicate transitions in context A. In context B, the transitions from states 2 to 4 and 3 to 5 are interchanged (light gray arrows). Bb) As panel Ab. Background shading indicates context (context A: white; context B: gray). Bc–Be) Actual hidden state, input spikes, networks spikes, and network belief; see panels Ac–Ae. Bf) Summary of network performance. “opt” shows performance of the optimal context-dependent Bayesian filter and “mix” a Bayesian filter where the transition rates are the mean rates over contexts A and B. The spiking network performs significantly better than the mixed Bayesian filter (paired t-test, p<0.001).

https://doi.org/10.1371/journal.pcbi.1003859.g008

Results from simulations of a particle filter circuit with 2000 neurons per state and an estimation sample size of 400 are shown in Figure 8Ac–Af, see Methods for details. Figure 8Af shows a comparison of the model performance (“model”) with the optimal Bayesian filtering (“opt”) and the optimal model that does not take temporal information into account (the Bayes estimate based on the latest observation; “inp”). The performance of the model was very close to optimal and much better than the non-temporal Bayes estimate. We furthermore tested the robustness of the network to variations of synaptic efficacies that determine the assumed transition rates of the random variable. In a control experiment, each individual synaptic weight from ensemble to ensemble was drawn from a log-normal distribution with mean and standard deviation . The resulting weights assumed values that were up to 10 times larger than the mean. Network performance with jittered efficacies was indistinguishable from the performance of the homogeneous network (Figure 8Af; “jit”). This robustness stems from two features of ENS-coding. First, as network belief is represented by ensemble rates, it is invariant to firing rate variations of individual neurons as long as the ensemble rate is preserved. Second, network computations are generally based on averages over ensemble activities, see eq. (14). Therefore variations in synaptic efficacies do not influence the result as long as the mean ensemble-to-ensemble weights are preserved. Note that this even holds for nonlinear link-functions, eq. (3).

Task class D: Context-dependent Bayesian filtering

In many important situations, the dynamics of a random variable changes in different contexts. Context cannot simply be formulated as a type of observation since observations just influence the probabilities of states for the given transition rates and not the dynamics themselves. Consider for example the estimation of the current body position in space. Here, an action such as forward movement can be considered as context since it increases the transition rates from any position to positions ahead. Thus, it changes the transition rates and not just the probability of a particular position. For possible contexts, consider a function with range that indicates the context at time . We define a context-dependent Markov chain as a Markov chain with state transition rates for each pair of distinct states . At each time , the chain evolves according to . Context-dependent Bayesian filtering determines the distribution over the current state for the given observations and context. Note that Bayesian filtering used in task class C is a special case with just a single context. Also note that we use the term context here with the specific meaning of additional information about the dynamics of the random variable.

We encode the current context by ensembles that provide contextual feedback to the circuit. Each context ensemble consists of context neurons . At time , only the context ensemble is active. The circuit architecture for particle filtering (Figure 4A) is extended as shown in Figure 9 to perform particle filtering for a hidden random variables with context-dependent dynamics. Layer consists of representations of the random variable, one for each context. Hence, for each context there are ensembles in this layer. In addition to excitatory synaptic connections originating from , these neurons are disinhibited by context neurons from the corresponding context ensembles. The disinhibitory effect of context is consistent with experimental data about neocortical circuits. There, context information from other cortical areas is believed to be provided through feedback connections. Those connections have abundant terminals in neocortical layer 1 where they recruit disinhibitory circuits [28], [29]. Formally, the membrane potential of , the m-th neuron in ensemble , is given by(15)where denotes the EPSP-filtered spike train of a neuron in ensemble of and denotes the efficacy of the connecting synapse. These efficacies are set proportionally to the transition rates in the corresponding context, see Table 2. The membrane potential of a neuron in is similar to the non-contextual case with the difference that each neuron is disinhibited by neurons that code for the same state in various contexts(16)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 9. Particle filter circuit architecture for task class D.

Extended circuit with ensembles (indicated by red and blue neurons respectively) and neurons per ensemble and two possible contexts. Ensembles in layer are duplicated for each context. These neurons receive context information via disinhibition from context neurons (yellow; only connections from context 1 shown for clarity). Disinhbition and lateral inhibition indicated by shortcuts as defined in Figure 4B, C. Arrows indicate efferent connections. A schematic overview of the circuit is shown in the inset.

https://doi.org/10.1371/journal.pcbi.1003859.g009

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Particle filter circuit equations for task class D.

https://doi.org/10.1371/journal.pcbi.1003859.t002

Due to disinhibition of layer by context ensembles, only those ensembles in the layer which correspond to the current context are active. The active neurons in this layer disinhibit neurons in in the same way as in the non-contextual case. Thus, in each individual context, a subcircuit is recruited that consists of the whole layer and the ensembles for the current context in layer . Since the weights from to ensembles for context in implement the dynamics of the random variable in that context, the circuit approximates the correct transition dynamics of the random variable in each context. The resulting membrane potential equations are summarized together with parameters in Table 2.

Particle filtering in ENS coding for self-localization.

Estimation of the body position in space (self-localization) is an essential ingredient of autonomous agents. In robotics, particle filtering is one of the most successful techniques for self localization [6]. Here, we demonstrate that particle filtering in ENS coding can be used for self-localization in environments with ambiguous evidence.

Every state of the considered hidden random variable corresponded to some position in the environment and transitions were possible between spatially adjacent states. The movement of the agent provided context for the particle filter such that movement in a particular direction enhanced transitions that point to that direction (Figure 10A). Sensory cues provided partial information about the current position. We simulated a two-chamber maze with a small opening that connects these chambers. The southern parts of the chambers gave rise to exactly the same observations, making it impossible to distinguish them without prior information (colored circles in Figure 10A). Observations in more northern parts were different and in the very north, no observations were experienced (corresponding for example to a dark corridor). Figure 10B shows the estimate of the network for a single trajectory. The model was started with a uniform prior distribution over all positions. In the southern terrain, the left and the right chamber cannot be distinguished, and accordingly, ensembles in both areas were active, indicating possible positions (t = 300 ms). In the more northern parts, observations disambiguate the current position, and activity in the right chamber ensembles ceased (t = 600, 900 ms). As the upper northern region was reached, no more evidence was provided from the environment (t = 1500, 2400 ms). Still, the network predicted movement to the right correctly thus utilizing movement information provided via context ensembles. As the right chamber was entered, the posterior was sharpened due to unambiguous sensory input. It remained single peaked even in southern parts of the chamber although these terrains produced ambiguous sensory input (t = 3300, 3800 ms).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 10. Self-localization through particle filtering in ENS coding.

A) Two-chamber maze (black lines) and transitions between states in various contexts. States are arranged on a 10×10 grid in the maze (crossing points of gray lines). Light gray lines indicate bidirectional state transitions with low transition rates (0.1 Hz). Dark gray arrows indicate transitions with high transition rates (3.5 Hz). Context is defined by movement direction (right, down, left, up). Colored circles indicate sensory evidence. Each color stands for one afferent neuron with a Gaussian spatial receptive field. The circle indicates the STD of the Gaussian. Note that the southern chambers give rise to identical observations. Observations are truncated at the height of the opening between the chambers such that no observations are experienced in the most northern parts. B) Network estimate of posterior probability (see color bar on the right for color code) for one trajectory through the maze (white trace; dot denotes current position) at different times. Spatial layout as in A. Various phases of the trajectory are shown: Uninformative prior knowledge (t = 10 ms), ambiguous estimates (t = 300 ms); disambiguation (t = 600, 900 ms); states without evidence (t = 1500, 2400 ms); unambiguous state estimation based on ambiguous evidence (t = 3300, 3800 ms).

https://doi.org/10.1371/journal.pcbi.1003859.g010

Task class E: Internal beliefs as context

In many tasks, the context variable is not explicitly available to the animal but rather has to be estimated from noisy evidence as well. For example, in many sequential tasks the current stage within the task can provide valuable context information which can be used to time actions or to decide when beliefs should be reset. In the computational framework considered here, this can be achieved by treating the context variable as a random variable that is estimated by a particle filter circuit from the given evidence . Estimated context is then utilized for the estimation of a random variable in a particle filter circuit with context.

In the tasks considered in task class D, the context was unambiguously given and all neurons of exactly one context ensemble had a firing rate larger than zero. This is not the case in task class E, since here context neurons encode a belief about the current context. The context-dependent particle filter should therefore deal with context in a graded manner. This is accomplished by a simple modification of how neurons in layer integrate context, see Methods. In the modified circuit, effective transition rates of the context-dependent filtering circuit are given as a linear mixture of the context-dependent transition rates , where each contributes approximately proportionally to the current belief in this context(17)

To demonstrate the viability of this approach, we reconsidered the ambiguous target task.

The ambiguous target task revisited.

The ambiguous target task has a clear sequential structure (initial fixation, spatial cue, memory epoch, color cue), see Figure 1A. The corresponding hidden variable c(t) – for which the current value is given by the momentary stage of the experimental trial – is shown in Figure 11Aa.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 11. Context-dependent Bayesian filtering in two successive trials of the ambiguous target task.

A) Represented random variables. Aa) Dynamics of a random variable that codes the current phase in a trial of the ambiguous target task (CHT: fixation; SC: spatial cue; MEM: memory cue; CC: color cue). Possible observations in each phase are indicated in boxes. Ab) Context-dependent Bayesian filtering is performed for a random variable with 16 hidden states corresponding to direction-color pairs as in Figure 6Aa. Gray lines indicate context-dependent transitions. All-to-all transitions are possible in the fixation phase (CHT). There are no transitions in other phases of a trial. Ac) The action readout layer infers a color-independent random variable by marginalization over color in each direction. B) Circuit structure. The circuit on the top (ensembles and ) performs Bayesian filtering on the random variable indicated in panel (Aa). It provides context for another particle filter circuit (middle gray box; ensembles and ) that generates a belief about the random variable indicated in Ab. An action readout layer is added (bottom gray box; ensembles ). C) Spike rasters from a simulation of two successive trials for afferent neurons (Ca), neurons in the particle filter circuit for the phase in the trial (Cb), and neurons in the action readout layer (Cc). Neurons in Cb are coding for the current phase of the trial (ordered from bottom to top: CHT, SC, MEM, and CC). Neuron ordering in Ca and Cc as in Figure 6.

https://doi.org/10.1371/journal.pcbi.1003859.g011

The modeled task is similar to the ambiguous target task considered above with the difference that several trials of the task are performed sequentially, i.e., a sequence of epochs (fixation, spatial cue, memory, color cue) is directly followed by the fixation epoch, indicating the start of a new trial, again followed by the spatial cue and so on. The difficulty of this task is that in the fixation epoch, the internal belief about the random variable that encodes the current color-direction pair is highly biased by the last trial. This bias is problematic since an optimal prior would assign equal probability to all color-direction pairs. In other words, during the fixation, should be reset. A reset can be accomplished by assuming that during fixation the dynamics of are such that can quickly change its state from any state to any other state (i.e., transition rates are high between all pairs of states, see Figure 11Ab). In other epochs however, the assumption is that the value of the random variable is fixed and does not change (transition rates are zero between all pairs of distinct states). We arrive at a context-dependent Bayesian filtering problem where the dynamics of depend on the current estimate of , the epoch in the trial.

We modeled the context-dependent reset of the internal belief by extending the circuit for the ambiguous target task in the following manner. The internal belief about the random variable that encodes the current color-direction pair (Figure 11Ab) is generated by a particle filter circuit. Context is provided to this circuit by an estimate of , the current epoch in the trial (Figure 11B). This estimate is performed by a particle filter circuit that receives the same sensory evidence but no context (task class C). In the context of the fixation-epoch, is believed to change its state rapidly (all-to-all transitions in the dynamics of the random variable, gray connections in Figure 11Ab). In the context of other epochs, the random variable is believed to be constant (no transitions between states, i.e., pure evidence integration). Finally, an action readout layer derives the belief over the rewarded action. In Figure 11C, a simulation for two successive trials of the ambiguous target task is shown. The epoch is correctly inferred by the particle filter circuit that provides the epoch-context (Figure 11Cb). This context influences information processing in the subsequent circuit for (Figure 11Cc). While information is retained during the memory epochs, the belief about the rewarded movement direction returns to an uninformative prior during the memory epoch at 1 s to 1.25 s. This example demonstrates that internally generated beliefs about random variables can act as valuable context in ENS coding.

Model simplifications and possible extensions

For consistency with the neural sampling approach, we used in this article rectangular shaped EPSPs. To test how deviations from rectangular EPSPs effect circuit performance, we performed simulations where all EPSPs were modeled as exponentially decaying EPSPs with decay time constant ms such that the integral over the EPSP is unchanged. The particle filter circuit was tested in the generic setup for task class C (see Figure 8A). Despite of its strong deviation from the rectangular shape, exponential EPSPs caused only a slight decrease of circuit performance (the percentage of incorrect state estimates was % as compared to % for rectangular EPSPs).

The model also uses particular forms of instantaneous lateral inhibition and disinhibition. Although both forms of inhibition have been reported to play crucial roles in cortical information processing [20]–[23], [28], [29], the exact circuitry and function of those inhibitory circuits is still unknown. The lateral inhibition used in our circuit model is however consistent with a recent study which showed that inhibition has a broader spatial selectivity than excitation in visual cortex of awake mice [31]. Such broader selectivity is expected in our model since lateral inhibition is common to all ensembles for a random variable. Apart from that, the assumption that inhibition follows excitation instantaneously is clearly a model simplification. A recent experimental study [32] revealed that cortical inhibition lags excitation by about 3 ms in anesthetized rats (the lag is possibly smaller in awake animals, see [31]). In order to test whether the circuit is tolerant to delayed inhibition, we performed control simulations where in addition to the use of exponential EPSP shapes, both lateral inhibition and disinhibition was delayed by 3 ms in the generic setup for task class C (see Figure 8A). We found that delayed lateral inhibition can lead to activity peaks as excitation arising from incoming evidence cannot be compensated rapidly. This can lead to unstable circuit activity when lateral inhibition tries to compensate these peaks in the delayed negative feedback loop. Instabilities can be avoided by reduction of the inhibitory drive, i.e., by reducing the lateral inhibition scaling in eq. (33). Simulations showed that with reduced inhibition scaling, the network tolerates delayed inhibition with a slight decrease of performance (the percentage of incorrect state estimates was % as compared to % without delay; ).

In ENS coding, each neuron is tuned to one value of a random variable . The encoded random variable may represent a specific feature relevant for some task. Many experiments show that such tunings exist in various cortical areas, such as the tuning of PMd neurons to potentially rewarded movement direction [3]. However, the random variable does not necessarily correspond to a single task-relevant feature. For example, the random variable encoded in the particle filter circuit for the ambiguous target task represents direction-color pairs (see Figure 6Aa). Therefore, neurons in this circuit are selective for both the spatial cue and the color cue. This mixed tuning helps to integrate the temporally separated cues. Mixed selectivity of neurons has been found in higher cortical areas such as prefrontal cortex, and its computational benefits have been highlighted in [33]. Hence, ENS coding is consistent with these findings. We note however that in the pure formulation of ENS coding, mixed selectivity to many task aspects is problematic since the number of ensembles necessary to encode all possible states over task dimensions grows exponentially with . One possibility to overcome exponential growth for large is to consider approximations schemes such as neglecting mixed configurations that are highly unlikely.

The important question how the parameters of the network could be attained by learning from experience is outside of the scope of this paper. However, some possible solutions to the learning problem can be sketched. For the particle filtering circuit, two classes of synaptic connections could be adapted through learning processes. First, synaptic efficacies from evidence neurons to neurons in layer . These efficacies encode the log-firing rates of evidence neurons for the given hidden state . It has been shown that hidden-cause representations that require such synaptic efficacies can be learned in spiking neural networks with lateral inhibition through spike-timing-dependent (STDP)-like synaptic plasticity rules [34], [35]. Hence, it seems quite feasible that the efficacies can be attained in an self-organized manner through STDP. The second type of connections, from to encode the dynamics of the random variable in terms of the rate of change from state to state . In other words, the synapse needs to track how often neuron is active after neuron (the synaptic efficacies needed for task class A are of a similar nature). Again, this tracking can be done by a temporal Hebbian learning rule. In particular, it has recently been shown in [36] that such temporal relationships can be learned through STDP-like learning rules in networks of spiking neurons that implement hidden Markov models. A more sophisticated learning approach that requires additional circuitry was outlined in [37]. Of course, these results do not immediately generalize to the architecture proposed in this article, and further studies are needed to prove the viability of such a learning approach. Finally, we note that the feature of the particle filter circuit (and ENS coding in general) that individual synaptic efficacies do not need to be adjusted to exact values as long as the mean efficacy between ensembles is correct (see Particle filtering in a generic setup for task class C) may prove advantageous for learning processes.

Experimentally testable predictions

We investigated the behavior of the model and compared it to experimental results. Our results so far indicate that ENS coding is consistent with a number of experimental studies. It is noteworthy that the lateral inhibition that is needed to stabilize network firing rates leads to the typical transient ensemble rate increases at stimulus-onset (see transient responses in Figure 5D and Figure 1C in Text S5).

While many laboratory tasks implement a variant of evidence integration (task class B), there is a lack of studies in the experimental literature for task classes D and E. We hypothesized in Task class E: Internal beliefs as context that the current values of important context variables are estimated in higher brain areas. Hence, one prediction of the model is that neural activities in such areas should not only be related to variables of primary interest, but also to context such as the current phase in a task with sequential structure, see Figure 11B,C. Evidence for such representations in monkey dorsolateral prefrontal cortex has been reported [38]. Our model also predicts that activity in neuronal ensembles that represent context should modulate the activity of decision-related ensembles in a manner that is fundamentally different from the impact of direct evidence. In particular, according to our circuit models in task classes D and E, context gates or modulates activity in such neurons. Task context has been shown to modulate neuronal activity in primate prefrontal cortex [39] as well as in various lower level visual areas including area MT [40], [41]. These findings are contrasted by studies that showed that task-context influenced noise-correlations but not firing rates in MT in a variant of the random-dot motion task [42]. The general term context of course subsumes many different types of contextual information in various quite different task settings, which may explain the discrepancies between the different studies. In this work, context is defined specifically as additional information about the dynamics of the random variable. Experimental setups where context information is indicative of the dynamics of task-relevant variables would help to elucidate how such information alters temporal processing in cortical circuits.

We proposed disinhibition as one possibility for context-dependent modulation, consistent with the experimental findings that disinhibitory circuits are recruited by feedback connections in neocortical layer 1 [28]. We note however that such effects could be implemented in cortical circuits by a number of mechanisms [18], [19].

We applied particle filtering – one of the most successful techniques for self localization in autonomous robots – in ENS coding to self-localization, a particularly important task for many animals. Whether self-localization in animals is solved in a similar manner is of course still unknown. There have however been studies which show that ambiguous sensory information can be resolved on the neuronal level in rodents [43], [44]. This indicates that the algorithm employed by the brain is in fact quite powerful. The particular implementation proposed here also implies that the spatial structure of the environment (i.e., possible transitions between locations in space) should be encoded in the synaptic weight matrix of particular neural circuits (edges and arrows in Figure 10A). Furthermore, our model predicts that information about motor events (movement) is treated by such circuits as context. Hence, these signals impact circuit activity quite differently from sensory evidence, see the discussion above. Evidence for nonlinear interaction of visual information and movement information in self-localization of mice has been reported recently [44].

Related work

Probabilistic population codes (PPC; [24]) have been suggested as one hypothesis how probability distributions could be coded in the spiking activity of neurons. In the PPC concept, one assumes that each neuron is (at least implicitly) linked to a stimulus via a tuning function. A hypothetical decoder would then apply Bayes rule to decode the stimulus distribution, making use of the tuning functions of the neurons. In ENS coding considered here, the neural ensembles produce samples from a distribution and a hypothetical decoder would just count spikes. Accumulation of evidence in LIP in a random-dot motion task has been modeled in the PPC framework [45]. In general, information can be accumulated in PPC simply by adding up activity from afferent neurons, given that this activity follows Poisson-like statistics. The model however assumes that the hidden variable is static. In fact, tracking of dynamic variables, such as those considered in task classes C–E, is hard to implement in the PPC framework [46].

Several spiking neural network models for Bayesian filtering have been proposed in the literature with very similar basic ideas to solve the problem [15], [47], [48]. Denève [47] proposed a model where a single integrate-and-fire neuron estimates the hidden state of a binary random variable with temporal dynamics. The model can only deal with binary variables, whereas our proposed model is not restricted in this respect. A similar model was proposed in [48]. There the assumption was that the continuous-valued random variable evolves according to a drift-diffusion process. Our implementation of Bayesian filtering in ENS coding complements this work by considering discrete-valued random variables with the assumption that the dynamics can be approximated by a continuous-time Markov chain.

Rao [15] considered Bayesian filtering in discrete time through a network of spiking neurons. This model was however not based on a rigorous coding scheme with respect to information transfer through spikes. The instantaneous firing rate of individual neurons was regarded as the distribution-encoding quantity and it was implicitly assumed that spikes can communicate this quantity in sufficient quality. In the current article, we base representations of beliefs on the spiking activity in the first place and propose ENS coding as a solution where the fidelity of representation is provided through ensemble activity. Our analysis identified how network properties such as the ensemble size, the maximal firing rate, and the membrane time constant influence the quality of the representation. In any case, the noise introduced by stochastic spiking cannot be neglected in general. We have demonstrated in computer simulations that still, temporal information processing on demanding tasks is possible in ENS coding.

We have argued that particle filtering with context is an important operation that is needed for example for self-localization. The current work shows that this extension of Bayesian filtering, that has not been considered in previous models, can easily be implemented in the ENS code.

Several non-spiking models for Bayesian filtering have been proposed previously [14], [49], [50]. We have based our circuit model for task classes B and C on well-known filtering equations [13] that also provided the basis for the rate-based model considered in [14]. Several conclusions can be drawn when comparing the model based on ENS coding considered here with the non-spiking model from [14]. First, we confirmed through computer simulations that quite demanding information processing tasks are possible with spiking neurons using ENS coding, despite of substantial noise introduced by stochastic spiking. Second, ensemble coding is clearly necessary for the tasks considered in this article if neuronal responses are stochastic. We used on the order of 1000 neurons per ensemble in the simulations. This number was not optimized, but ensemble sizes below 100 are not sufficient, for example in the ambiguous target task (see Dataset S1). Third, besides the complications that stochastic spike codes introduce, we have shown that ENS coding also has some positive effects. We found that multiplicative operations can be replaced by gating of neuronal activity in ENS coding, for example through synaptic gating or through disinhibition. This property of ENS coding provides an attractive alternative to previously proposed solutions for the unavoidable demand of nonlinear processing in Bayesian filtering, such as multiplicative interaction of synaptic inputs [14], [50], or the use of precise dendritic nonlinearities [15]. Finally, the use of a spiking model enabled us to directly compare model characteristics to experimental data. We found that the model is consistent with quite diverse experimental results [1]–[3], [25].

Conclusions

Sample-based representations of probability distributions provide an attractive framework for modeling probabilistic inference on static evidence in cortical networks. We have shown that ensemble-based neural sampling enables cortical networks to perform also powerful context-dependent temporal inference. Hence, our model provides a new and theoretically founded basis for understanding temporal probabilistic computations in various higher-level cortical areas.

Spike trains, EPSP shapes, and EPSP-filtered spike trains

Here, we define EPSP-filtered spike trains and the shape of EPSPs used throughout the article. We denote the spike-train of a neuron as the sum of Dirac delta functionswhere denotes the j-th spike-time of neuron . We define the EPSP-filtered spike train as

(18)Here denotes the EPSP shape produced by a spike of a presynaptic neuron. We use in this article rectangular EPSP shapes of the form(19)where denotes the Heaviside step function and is the length of the EPSP. For some control simulations, we use exponentially decaying EPSP shapes

(20)

ENS code and filtered probability distributions

Assuming rectangular EPSP shapes of length , the filtered probability denoted as is the average probability for state in time , that is(21)

We denote the ensemble average of a random variable by . Consider a given fixed temporal evolution of the probability . In ENS coding, we demand that neural activities are such that the mean of the estimator is equal to the temporally filtered probability of that state at time t, that is(22)

Mean and variance of the estimator

Here we show that eq. (1) is an unbiased estimator of the probability of state at time filtered by the EPSP, given that there is at least one spike in the integration window. Additionally we compute the variance of the estimator.

Consider a given fixed temporal evolution of the probability . We first consider the mean of for a given number of spikes , written as :

Given a spike at time t, the probability that it was elicited in ensemble is , where denotes the total network rate. Since the total network rate is constant and each spike is drawn independently, to count the spikes in different ensembles in a time window , we can replace each inhomogeneous Poisson process by a homogeneous process in that time window with rate , where denotes the temporally filtered probability of the state, see eq. (21). Each individual spike in the time window originates from ensemble with probability . Hence, for a given , is drawn from a binomial distribution . It follows that and . Hence, , and the estimator is unbiased.

We now turn to the variance of . We define and for notational convenience. From , we obtain . By the law of total variance, we have

The second summand is 0 since is independent of . We thus obtain

Here, is Poissonian with intensity . Inserting the Poisson density, we obtain

The approximation is excellent if is large, i.e., a small number of spikes over all ensembles within is very unlikely.

Influence of the firing rate on the represented distribution

We provide here the proof for eq. (2). We assume that an antiderivative exists for all rates (which is satisfied for example if the rates are continuous or sums of Heaviside step functions). The probability mass of a population is given by(23)where is the spike train of neuron as defined above. The neurons spike in a Poissonian manner with continuous rates . Hence we obtain for the mean over realizations of spike trains(24)It follows(25)

Task class A: Simple probabilistic dependencies

We first show that the expected value of the estimator is equal to the posterior distribution (4) for membrane potentials (5) and weights (6). We will then derive the variance of an alternative estimator of the posterior distribution.

Consider two distributions and over random variables and respectively such that the desired posterior is given by eq. (4). is coded by neurons through ENS coding. We denote the estimation sample size of the ensembles for by for clarity. Consider a circuit with neurons and ensemble size that should represent the posterior. The membrane potentials are given by for . For given EPSP-filtered spike trains , this leads to the firing rate for neuron (26)

Averaging over realizations of spike trains in the posterior population, we obtainwhere denotes the average for given activities in ensembles . Taking also the average over realizations of spike trains in ensembles , this evaluates to

We assume for simplicity that is constant over time, and obtainwhere we defined . This shows that the represented probability of the circuit is the posterior probability in the mean with an estimation sample size of .

Task class B: Evidence integration

Disinhibition

We formally define the influence of disinhibition arising from neurons on the membrane potential of a neuron as(32)where is some baseline inhibition and is some threshold. In other words, the disinhibited neuron is released from baseline inhibition if the neurons were recently active enough to overcome the threshold . We used in this article a threshold of . For rectangular EPSPs, this results in disinhibition whenever . In our circuit model, the baseline inhibition is strong enough to suppress any activity when inhibited.

Lateral inhibition

Lateral inhibition in layer is given by(33)

It corrects for excessive spiking activity, i.e., when . The constant determines how strongly this correction influences the membrane potential. We included the operator to model a purely inhibitory population. This means that too low firing rates are not compensated by this mechanism. This is possible, since all other contributions to the membrane potential are non-negative if weights are shifted to non-negative values. In particular, the term in the biases (see Table 1) leads to a quick recovery of the ensemble firing rate.

Strong lateral inhibition at low potentials could be cut off by the -operator in equation (28). That would result in unequal inhibition values for different ensembles. This can be avoided by bounding inhibition to a maximum of (note that inhibition is bounded due to the finite size of the network). But our simulations indicate that even with larger values, the circuit performs very well.

General details on computer simulations

All simulations were performed in discretized time with a time step of 0.5 ms. The duration of EPSPs was set to ms in all simulations. The delay for lateral inhibition was set to 0.5 ms. The initialization of network activity for the particle filter circuits was performed as follows. Let denote the distribution that the network should represent at the beginning of a simulation. Spikes were drawn in both layers and distributed in the time interval such that at time , the represented probability represented in the mean. More precisely, a spike of neuron in was assumed with probability . If there was a spike for this neuron, its exact timing was drawn from a uniform distribution in the interval . Simulation of the network began at t = 0 ms where the EPSPs of theses spikes were taken into account when computing the membrane potentials of neurons during the initial phase of the simulation. The initially represented distribution was assumed uniform over all states if not otherwise noted.

All simulation code is available online as Dataset S1. Details of individual simulations are discussed in the following.

Computer Simulations for task class B: Evidence integration

Action readouts from state variables

We provide here details on the readout layer used to model the ambiguous target task and the random-dot motion task. An application of the feedforward circuit discussed in Simple probabilistic dependencies is inference over rewarded actions based on the current belief about the state of a random variable . In the context of reward-based action selection, one wants to estimate the probability that a given action leads to reward (we consider here binary rewards for simplicity). If the probability of reward for action in state is known, then the posterior distribution over actions for the current state distribution can be inferred as(34)see below. In the ambiguous target task, is 1 if the action (movement direction) matches the direction of the spatial cue in the color of the color cue, and 0 otherwise. Eq. (34) defines a direct probabilistic relation between the action and the state. Hence, a layer of neurons as described in Task class A computes the posterior distribution. We refer to such a layer as an action readout layer.

We discuss this now more formally. Consider a random variable with range , a random variable over possible actions , and a binary random variable that indicates whether a reward occurs. Assuming that the joint probability factorizes to , one can infer future actions that will lead to a reward for the given distribution over states by:(35)

We want to infer future actions that will lead to a reward for the current state . Assuming a uniform prior over actions, this simplifies to(36)

Assume that the distribution is given by ensembles in a sample-based representation with estimation sample size . This defines an operation as discussed above under Task class A: Simple probabilistic dependencies. Hence, a circuit consisting of ensembles and neurons per ensemble samples from the posterior distribution (36) with estimation sample size if the membrane voltages of the neurons are given bywith weights .

The ambiguous target task

Stimuli were coded by 19 ensembles of afferent neurons, where each ensemble consisted of 20 neurons. One ensemble coded for the fixation cross, and two for the two colors of the color cue. Eight ensembles coded for each position of a red spatial cue and eight for each position of a blue color cue. First, the fixation cross-stimulus was presented for 100 ms. The spatial cue followed for 250 ms. Here, two ensembles of afferent neurons were active, one for each color of the spatial cue. In the 250 ms memory epoch, only the fixation cross was present. After the memory epoch, the color cue appeared, and the corresponding ensemble of afferent neurons (for red or blue color cue) was active. Afferent neurons spiked at a baseline rate of 0.1 Hz and at a rate of 5 Hz when they were active.

For synaptic weights from afferent neurons, we used for observations that are possible in state . For example, in a state coding for “movement to 90 degrees with red color cue”, a red spatial cue at 90 degrees, a blue spatial cue at 270 degrees, and a fixation cross are possible. All other were set to the baseline firing rate of 0.1 Hz. The lateral inhibition scaling was .

The action readout layer consisted of 100 neurons for each of the 8 possible actions (movement to one of eight directions). Considering eq. (4), we identified the random variable with the random variable estimated in the particle filter circuit. The hidden states of this variable are the direction-color pairs for 8 possible directions and 2 colors . The random variable has 8 states encoding that movement in direction leads to a reward. This random variable can be computed through marginalization over color . Hence, the conditional probabilities were given by for and otherwise. With these conditionals, we obtained the synaptic weights through eq. (6) with , leading to an estimation sample size of 80 in the action readout layer.

For Figure 1, spiking activity was averaged over 50 successful trials (87 successful trials out of 100), temporally (10 ms running average) and spatially (mean activity of 10 neighboring neurons for neurons ordered by their preferred direction).

Continuous time Markov chains

A continuous time Markov chain is characterized by transition rates for each pair of distinct states, i.e., for and . Assume that at some time the state is . To sample a sequence of states from the Markov chain starting at , a Poisson process with rate is started for each state . The time of the next state transition is given by the first event in these processes and the next state is given by the index of the process that produced the event. After a transition, the chain starts afresh at the transition time.

Task class C: Bayesian filtering

We show that the membrane potentials given in Table 1 approximate the set of differential equations (13). These equations can be formulated as two sets of coupled equations, one for evidence integration and one for prediction of dynamics:(37)(38)where we assume identical initial conditions for and . is represented by and the change of the expected probability masses corresponds to the changes in eq. (37) as shown above.

We show here that the membrane potentials given in eq. (14) lead to changes in the expected probability masses of that approximate those of eq. (38). Assume that the membrane potentials evolve according to eq. (14) with weights where we defined . For given spikes in layer , the summed firing rates in layer evaluate to

The change of the expected probability masses in is

The last term is due to EPSPs that end at time t which needs to be compensated. It is approximately compensated by the second to last term since under the assumption that the expected firing rate (i.e., the represented probability distribution) changes slowly on the time scale of the EPSP. Altogether we obtainas needed.

Computer Simulations for task class C: Bayesian filtering

Computer Simulations for task class D: Context-dependent Bayesian filtering

Task class E: Graded integration of uncertain context information

In task class E, it is advantageous to deal with context in a graded manner. This is achieved by a simple modification the context-dependent filtering circuit from task class D. In the modified circuit, the membrane potential of neurons in is given bywhere denotes the EPSP-filtered spike train of the neuron in the ensemble that represents the distribution over contexts. In comparison to the equation given in Table 2, each neuron in is disinhibited by a single context neuron. This leads to an approximate linear mixture of the context-dependent transition rates

where is the effective transition rate for the context-dependent filtering circuit at time and is the estimated posterior for the context variable at time .

Computer simulations for task class E: The ambiguous target task revisited

State sequences and spike trains of evidence neurons were generated as in the simulation for the ambiguous target task, with the exception that the fixation phase lasted for 250 ms. Circuit parameters for the context-dependent filtering of and the action readout layer were identical to those in the simulation for the ambiguous target task (1000 neurons per sate; estimation sample size 400; ). In the fixation-context, transition rates between all pairs of states were 1 Hz.

A particle filter circuit (without context) was employed to estimate , the current phase of the trial. The basic parameters for this circuit were identical to the parameters of the context-dependent filtering circuit (1000 neurons per sate; estimation sample size 400; ). Other parameters for this circuit were as follows. Synaptic weights to were set according to transition rates that were assumed to be 5 Hz between states where a transition is possible and 0 Hz between other states. Synaptic weights from evidence neurons to the evidence layer were set according to the following observation rates (see also Figure 11). A fixation cross in the CHT state or in the MEM state: 5 Hz; Any spatial cue in the SC state: 5/16 Hz; Any color cue in the CC state: 2.5 Hz; For other observation rates, a baseline of 0.1 Hz was assumed.