Volatility of sensory contingencies and the adaptation of cognitive systems

We live in a fundamentally volatile world for which our cognitive system has to constantly adapt. In particular, this volatility may be generated by processes with different time scales. Imagine for instance you are a general practitioner and that you usually report an average number of three persons infected by flu per week. However, this rate is variable and over the past week you observe that the rate increased to ten cases. In consequence, two alternative interpretations are available: the first possibility is that there is an outbreak of flu and one should then estimate its incidence (i.e. the rate of new cases) since the inferred outbreak’s onset, in order to quantify the infection rate specific to this outbreak, but also to update the value of the probability of a new outbreak at a longer time scale. Alternatively, these cases are “unlucky” coincidences that originate from the natural variability of the underlying statistical process which drive patients to the doctor, and which are instances drawn from the same stationary random process. In that option, it may be possible to readjust the estimated baseline rate of infection with this new data. This example illustrates one fundamental problem with which our cognitive system is faced: when observing new sensory evidence, should I stay and continue to exploit this novel data with respect to my current beliefs about the environment’s state or should I go and explore a new hypothesis about the random process generating the observations since the detection of a switch in the environment?

This uncertainty in the environment’s state is characterized by its volatility which by definition measures the temporal variability of the sufficient parameters of a random variable. Such meta-analysis of the environment’s statistical properties is an effective strategy at a large temporal scale level, as that for the flu outbreak of our example, but also at all levels which are behaviorally relevant, such as contextual changes in our everyday life. Inferring near-future states in a dynamic environment, such that one can prepare to act upon them ahead of their occurrence—or at least forming beliefs as precise as possible about a future environmental context—is an ubiquitous challenge for cognitive systems [1]. In the long term, how the human brain dynamically manages this trade-off between exploitation and exploration is essential to the adaptation of the behavior through reinforcement learning [2].

In controlled psychophysical experimental settings which challenge visual perception or sensorimotor associations, such adaptive processes have been mostly put in evidence by precisely analyzing the participants’ behavior in a sequence of experimental trials. These typically highlight sequential effects at the time scale of several seconds to minutes or even hours in the case of the adaptation to a persistent sensorimotor relation. Indeed, stimulus history of sensory events influences how the current stimulus is perceived [3–7] and acted upon [8–11]. Two qualitatively opposite effects of the stimulus history have been described: negative (adaptation), and positive (priming-like) effects. Adaptation reduces the sensitivity to recurrently presented stimuli, thus yielding a re-calibrated perceptual experience [12–14]. On the other hand, priming is a facilitatory effect that enhances the identification of repeated stimuli [15, 16]: in sensorimotor control, the same stimulus presented several times could indeed lead to faster and more precise responses. Interestingly, priming effects are sometimes paralleled by anticipatory motor responses which are positively correlated with the repetition of stimulus properties. A well-known example of this behavior are anticipatory smooth eye movements (aSPEM or shortly, anticipatory pursuit), as we will illustrate in the next section.

Overall, the ability to take into account statistical regularities in the event sequence appears as a fundamental ability for the adaptive behavior of living species. Importantly, few studies have addressed the question of whether the estimate of such regularities is explicit, and whether such explicit reports of the dynamic statistical estimates would eventually correlate with the measures of behavioral adaptation or priming. Here we aim at investigating this question in the specific case of the processing of a target’s motion direction. In addition, we attempt to palliate the lack of a solid modeling approach to best understand the computation underlying behavioral adaptation to the environment’s statistics, and in particular how sequential effects are integrated within a hierarchical statistical framework.

Bayesian inference offers an effective methodology to deal with this question. Indeed, these methods allow to define and quantitatively assess a range of hypotheses about the processing of possibly noisy information by some formal agents [17–19]. A key principle in the Bayesian inference approach is to introduce so-called latent variables which explicitly represent different hypotheses by the agent and how these may predict experimental outcomes. Each hypothesis defines different weights in the graph of probabilistic dependencies between variables (for instance between the number of patients at a practitioner and the reality of a flu pandemic). Then, using the rules of probability calculus and knowing incoming measurements, one can progressively update beliefs about the latent variables, and eventually infer the hidden structure underlying the received inputs [20, 21]. For instance, using Bayes’s rule, one can combine the likelihood of observations given a given generative model and the prior on these latent variables [22] such that beliefs about latent variables may be represented as probabilities. Of particular interest for us is the possibility to quantitatively represent in this kind of probabilistic model the predictive and iterative nature of a sequence of events. Indeed, once the belief about latent variables is formed from the sensory input, this belief can be used to update the prior over future beliefs [23]. In such models, the comparison between expectations and actual data leads to continuous updates of the estimates of the latent variables, but also of the validity of the model. There are numerous examples of Bayesian approaches applied to the study of the adaptation to volatility. For instance, Meyniel et al [24] simulated a hierarchical Bayesian model over five previously published datasets [25–29] in the domain of cognitive neuroscience. Here we focus on an extension of this approach to the study of motion processing and eye movements.

Anticipatory Smooth Pursuit Eye Movements (aSPEM)

Humans are able to accurately track a moving object with a combination of saccades and Smooth Pursuit Eye Movements (for a review see [30]). These movements allow us to align and stabilize the object on the fovea, thus enabling high-resolution visual processing. This process is delayed by different factors such as axonal transduction, neural processing latencies and the inertia of the oculomotor system [31]. When predictive information is available about target’s motion, an anticipatory Smooth Pursuit Eye Movement (aSPEM or shortly, anticipatory pursuit) is generated before its appearance [32–34] thereby reducing visuomotor latency [35]. Moreover, some experiments have demonstrated the existence of prediction-based smooth pursuit maintenance during the transient disappearance of a moving target [36–38] and even predictive acceleration or deceleration during visual tracking [37, 39]. Overall, although the initiation of smooth pursuit eye movements is almost always driven by a visual motion signal, it is now clear that smooth pursuit behavior can be modulated at different stages by extra-retinal, predictive information even in the absence of a direct visual stimulation [40]. Several functional and computational models have been proposed in the literature for the different forms of prediction-based smooth eye movements, such as zero-lag tracking of a periodic target [41] or pursuit maintenance during target occlusion [39]. More recently an effort has been made to provide a more general theoretical framework, which is based on Bayesian inference and the reliability-based cue combination. Although the mapping of this theoretical framework onto neuronal functions remains to be elucidated, it has the clear advantage of generality, as for instance, it would encompass all forms of smooth pursuit behavior, including prediction-based and visually-guided tracking [42–45]. Here, we present a model extending this recent theoretical effort to include the adaptivity to a volatile environment.

Experience-based anticipatory pursuit behavior is remarkable in different aspects. First, its buildup is relatively fast, such that only a few trials are sufficient to observe the effects of specific regularities in the properties of visual motion, such as speed, timing or direction [10, 44, 46]. Second, it is a robust phenomenon, which has been observed on a large population of human participants and even in non-human primates (for a recent review see [47]). Note also, that human participants seem to be largely unaware of this behavior (as inferred from informal questioning). Finally, this kind of behavior has proven to be exquisitely sensitive to the probabilistic properties of the sensorimotor context.

Typically, anticipatory pursuit is observed after a temporal cue and before target motion onset [33, 34, 46]. In previous studies [11, 48], we have analyzed how forthcoming motion properties, such as target speed or direction, can be anticipated with coherently oriented eye movements. We have observed that the amplitude of anticipation, as measured by the mean anticipatory eye velocity, increases when the target repeatedly moves in the same direction. In particular, the mean anticipatory eye velocity is linearly related to the probability of motion’s speed or direction. These results are coherent with findings by other groups [46, 49–51] and they indicate that anticipatory pursuit behavior is potentially a useful marker to study the internal representation of motion expectancy, and in particular to analyze how such expectancy is dynamically modulated by probabilistic contingencies in shaping oculomotor behavior.

Contributions

The goal of this study is to generalize the adaptive process observed in anticipatory pursuit [48, 51] to more ecological settings and also to broaden its scope by showing that such adaptive processes occur also at an explicit level. We already mentioned that by manipulating the probability bias for target motion direction, it is possible to modulate the strength (direction and mean velocity) of anticipatory pursuit. This suggests that probabilistic information about direction bias may be used to inform the internal representation of motion prediction for the initiation of anticipatory movements. However, previous studies have overlooked the importance to design a realistic generative model to dynamically manipulate the probability bias and generate an ecologically relevant input sequence of target directions. A possible confound comes from the fact that previous studies have used fixed-lengths sequences of trials, stacked in a sequence of conditions defined by the different probability biases. Indeed, observers may potentially pick up the information on the block’s length to predict the occurrence of a switch (a change in probability bias) during the experiment. Second, we observed qualitatively that following a switch, the amplitude of anticipatory pursuit velocity changed gradually, consistently with other adaptation paradigms [52–54]. The estimate of the characteristic temporal parameters for this adaptation mechanism may become particularly challenging in a dynamic context, where the probabilistic contingencies vary in time in an unpredictable way. Finally, whether and how the information processing underlying the buildup of anticipatory pursuit and its dynamics is linked to an explicit estimate of probabilities is still largely unknown.

To assess the dynamics of the adaptive processes which compensate for the variability within sensory sequences, one may generate random sequences of Target Directions (TDs) using a dynamic value for the probability bias p = Pr(TD is ‘right’), with a parametric mechanism controlling for the volatility at each trial. In the Hierarchical Gaussian Filter model [55], for instance, volatility is controlled as a non-linear transformation of a random walk (modeled itself by a Brownian motion with a given diffusion coefficient). Ultimately, this hierarchical model allows to generate a sequence of binary choices where volatility is controlled by a specific random variable which fluctuates in time according to some probabilistic law. Such a forward probabilistic model is invertible using some simplifying assumptions and allows to extract a time-varying inference of the agent’s belief about volatility [56]. Herein, to explicitly analyze the effect of history length, we rather extend the protocol of [48] such that the probability bias is still fixed within sub-blocks but that these sub-blocks have variable lengths, that is, by introducing switches occurring at random times. Therefore, similarly to [57], we use a model for which the bias p in target direction varies according to a piecewise-constant function. We expect that within each of these sub-blocks that we defined, the uncertainty about of the value of p will progressively decrease as we accumulate trials. In addition, the range of possible biases was finite (p ∈ {0, .1, .25, .5, .75, .9, 1}) in our previous study. In the present work, we also extend the paradigm by drawing p as a continuous random variable within the whole range of possible probability biases (that is, the segment [0, 1]).

As a summary, we first draw random events (that we denote as “switches”) with a given mean frequency (the “hazard rate”) and which controls the strength of the volatility. Second, the value p of the bias only changes at the moment of a switch, independently of the previous bias’ value and it is stationary between two switches, forming what we call an “epoch”. Third, target direction is drawn as a Bernoulli trial using the current value of p. Such a hierarchical structure is presented in Fig 1A, where we show the realization of the target’s directions sequence, the trajectory of the underlying probability bias (hidden to the observer), and the occurrences of switches. Mathematically, this can be considered as a three-layered hierarchical model defining the evolution of the model at each trial t as the vector . At the topmost layer, the occurrence of a switch (1 for true, 0 for false) is drawn from a Bernoulli trial parameterized by its hazard rate h (as the frequency of occurrence for each trial). The value of thus gives the average duration (in number of trials) between the occurrence of two switches. In the middle layer, the probability bias p of target direction is a random variable that we define as . It is chosen at random from a prior distribution at the moment of a switch, and else it is constant until the next occurrence of a switch. The prior distribution can be for instance the uniform distribution on [0, 1] or Jeffrey’s prior (see Appendix). Finally, a target moves either to the left or to the right, and we denote this variable (target direction, TD) as . This direction is drawn from a Bernoulli trial parameterized by the direction bias . In short, this is described according to the following equations: (1) In this study, we generated a sequence of 600 trials, and there is by construction a switch at t = 0 (that is, ). In addition, we imposed in our sequence that a switch occurs after trial numbers 200 and 400, in order to be able to compare adaptation properties across these three different trial blocks. With such a three-layered structure, the model generates the randomized occurrence of switches, itself generating epochs with constant direction probability and finally the random sequence of Target Direction (TD) occurrences at each trial. This system of three equations defined in Eq 1 defines the Binary Switching model which we used for the generation of experimental sequences presented to human participants in the experiments. We will use that generative model as the basis for an ideal observer model equipped to invert that generative model in order to estimate the time-varying probability bias for a given sequence of observations (TDs). The comparison of human behavior with the ideal observer model’s predictions will allow us to test it as a model for the adaptation of human behavior to the environment’s volatility.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Smooth pursuit eye movements and explicit direction predictions in a volatile switching environment.

We test the capacity of human participants to adapt to a volatile environment. (A) We use a 3-layered generative model of fluctuations in target directions (TD) that we call the Binary Switching model. This TD binary variable is chosen using a Bernoulli trial of a given probability bias. This probability bias is constant for as many trials until a switch is generated. At a switch, the bias is chosen at random from a given prior. Switches are generated in the third layer as binary events drawn from a Bernoulli trial with a given hazard rate (defined here as 1/40 per trial). We show one realization of a block of 200 trials. (B) The eye-movements task was an adapted version of a task developed by [48]. Each trial consisted of sequentially: a fixation dot (FIX, of random duration between 400 and 800 ms), a blank screen (GAP, of fixed duration of 300 ms) and a moving ring-shaped target (TARGET, with 15°/s velocity) which the observers were instructed to follow. The direction of the target (right or left) was drawn pseudo-randomly according to the generative model defined above. (C) In order to titrate the adaptation to the environmental volatility of target direction at the explicit and conscious level, we invited each observer to perform on a different day a new variant of the direction-biased experiment, where we asked participants to predict, before each trial, their estimate of the forthcoming direction of the target. As shown in this sample screenshot, this was performed by moving a mouse cursor (black triangle) on a continuous rating scale between “Left”, to “Unsure” and finally “Right”.

https://doi.org/10.1371/journal.pcbi.1007438.g001

This paper is organized in five parts. After this introduction where we presented the motivation for this study, the next section will present an inversion of the (forward) binary switching generative model, coined the Binary Bayesian Change-Point (BBCP) model. To our knowledge, such algorithm was not yet available, and we will here provide with an exact analytical solution by extending previous results from [58] to the binary nature of data in the Binary Switching model presented above (see Eq 1). In addition, the proposed algorithm is biologically realistic as it uses simple computations and is online, that is, all computations on the sequence may be done using solely a set of variables available at the present trial, compactly representing all the sequence history seen in previous trials. We will also provide a computational implementation and a quantitative evaluation of this algorithm. Then, we will present the analysis of experimental evidence to validate the generalization of previous results with this novel switching protocol. In order to understand the nature of the representation of motion regularities underlying adaptive behavior, we collected both the recording of eye movements and the verbal explicit judgments about expectations on motion direction. In one session, participants were asked to estimate “how much they are confident that the target will move to the right or left in the next trial” and to adjust the cursor’s position on the screen accordingly (see Fig 1C). In the other experimental session on a different day, we showed the same sequence of target directions and recorded participants’ eye movements (see Fig 1B). Combining these theoretical and experimental results, a novelty of our approach is to use the BBCP agent as a regressor which will allow us to match experimental results and to compare its predictive power compared to classical models such as the leaky integrator model. Hence, we will show that behavioral results match best with the BBCP model. In the following section, we will synthesize these results by inferring the volatility parameters inherent to the models by best-fitting it to each each individual participant. This will allow the analysis of inter-individual behavioral responses for each session. In particular, we will test if one could extract observers’ prior (preferred) volatility, that is, a measure of the dynamic compromise between exploitation (“should I stay?”) and exploration (“should I go?”) for the two different sessions challenging predictive adaptive processes both at the implicit and explicit levels. Finally, we will summarize and conclude this study and offer some perspectives for future work.

Binary Bayesian Change-Point (BBCP) detection model

As we saw above, Bayesian methods provide a powerful framework for studying human behavior and adaptive processes in particular. For instance, [55] first defined a multi-layered generative model for sequences of input stimuli. By inverting this stochastic forward process, they could extract relevant descriptors at the different levels of the model and fit these parameters with the recorded behavior. Here, we use a similar approach, focusing specifically on the binary switching generative model, as defined in Eq 1. To begin, we define as a control a first ideal observer, the leaky integrator (or forgetful agent), which has an exponentially-decaying memory for the events that occurred in the past trials. This agent can equivalently be described as one which assumes that volatility is stationary with a fixed characteristic frequency of switches. Then, we extend this model to an agent which assumes the existence of (randomly occurring) switches, that is, that the agent is equipped with the prior knowledge that the value of the probability bias may change at specific (yet randomly drawn) trials, as defined by the forward probabilistic model in Eq 1.

Binary Bayesian Change-Point (BBCP) detection model.

There is a crucial difference between the forgetful agent presented above and an ideal agent which would invert the (generative) Binary Switching model (see Eq 1). Indeed, at any trial during the experiment, the agent may infer beliefs about the probability of the volatility which itself is driving the trajectory of the probability bias . Knowing that the latter is piece-wise constant, an agent may have a belief over the number of trials since the last switch. This number, that is called the run-length r^t [58], is useful in two manners. First, it allows the agent to restrict the prediction of only based on those samples produced since the last switch, from t − r^t until t. Indeed, the samples which occurred before the last switch were drawn independently from the present true value and thus cannot help estimating the latter. As a consequence, the run-length is a latent variable that captures at any given trial all the hypotheses that may be occurring. Second, it is known that for this estimate, the precision (that is, the inverse of variance) on the estimate grows linearly with the number of samples: The longer the run-length, the sharper the corresponding (probabilistic) belief. We have designed an agent inverting the binary switching generative model by extending the Bayesian Change-Point (BCP) detection model [58]. The latter model defines the agent as an inversion of a switching generative model for which the observed data (input) is Gaussian. We present here an exact solution for the case of the Binary Switching model, that is, for which the input is binary (here, left or right).

In order to define in all generality the change-point (switch) detection model, we will initially describe the fundamental steps leading to its construction, while providing the full algorithmic details in Appendix. The goal of predictive processing at trial t is to infer the probability of the next datum knowing what has been observed until that trial (that we denote by ). This prediction uses the agent’s prior knowledge that data is the output of a given (stochastic) generative model (here, the Binary Switching model). To derive a Bayesian predictive model, we introduce the run-length as a latent variable which gives to the agent the possibility to represent different hypotheses about the input. We therefore draw a computational graph (see Fig 2A) where, at any trial, an hypothesis is formed on as many “nodes” than there are run-lengths. Note that run-lengths may be limited by the total number of trials t. As a readout, we can use this knowledge of the predictive probability conditioned on the run-length, such that one can compute the marginal predictive distribution: (4) where is the probability of the Bernoulli trial modeling the outcome of a future datum , conditioned on the run-length and is the probability for each possible run-length given the observed data. Note that we know that, at any trial, there is a single true value for this variable r^t and that thus represents the agent’s inferred probability distribution over the run-length r. As a consequence, is scaled such that .

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Binary Bayesian Change-Point (BBCP) detection model.

(A) This plot shows a synthesized sequence of 13 events, either a leftward or rightward movement of the target (TD). Run-length estimates are expressed as hypotheses about the length of an epoch over which the probability bias was constant, that is, the number of trials since the last switch. Here, the true probability bias switched from a value of .5 to .9 at trial 7, as can be seen by the trajectory of the true run-length (blue line). The BBCP model tries to capture the occurrences of a switch by inferring the probability of different possible run-lengths. At any new datum (trial), this defines a Hidden Markov Model as a graph (trellis), where edges indicate that a message is being passed to update each node’s probability (as represented by arrows from trial 13 to 14). Black arrows denote a progression of the run-length at the next step (no switch), while gray lines stand for the possibility that a switch happened: In this case the run-length would fall back to zero. The probability for each node is represented by the grey scale (darker grey colors denote higher probability) and the distribution is shown in the inset for two representative trials: 5 and 11. Overall, this graph shows how the model integrates information to accurately identify a switch and produce a prediction for the next trial (e.g. for t = 14). (B) On a longer sequence of 200 trials, representative of a trial block of our experimental sequence (see Fig 1A), we show the actual events which are observed by the agent (TD), along with the (hidden) dynamics of the true probability bias P_true (blue line), the value inferred by a leaky integrator (P_leaky, orange line) and the results of the BBCP model in estimating the probability bias P_BBCP (green line), along with .05 and .95 quantiles (shaded area). This shows that for the BBCP model, the accuracy of the predicted value of the probability bias is higher than for the leaky integrator. Below, we show the belief (as grayscales) for the different possible run-lengths. The green and orange line correspond to the mean run-length which is inferred, respectively, by the BBCP and leaky models: Note that in the BBCP, while it takes some trials to detect switches, they are in general correctly identified (transitions between diagonal lines) and that integration is thus faster than for the leaky integrator, as illustrated by the inferred value of the probability bias.

https://doi.org/10.1371/journal.pcbi.1007438.g002

With these premises, we define the BBCP as a prediction / update cycle which connects nodes from the previous trial to that at the current trial. Indeed, we will predict the probability at each node, knowing either an initial prior, or its value on a previous trial. In particular, at the occurrence of the first trial, we know for certain that there is a switch and initial beliefs are thus set to the values and ∀r > 0, . Then, at any trial t > 0, as we observe a new datum , we use a knowledge of at trial t − 1, the likelihood and the transition probabilities defined by the generative model to predict the beliefs over all nodes: (5) In the computational graph, Eq 5 corresponds to a message passing from the nodes at time t − 1 to that at time t. We will now detail how to compute the transition probabilities and the likelihood.

First, knowing that the data is generated by the Binary Switching model (see Eq 1), the run-length is either null at the moment of a switch, or its length (in number of trials) is incremented by 1 if no switch occurred: (6) This may be illustrated by a graph in which information will be represented at the different nodes for each trial t. This defines the transition matrix Pr(r^t|r^t−1) as a partition in two exclusive possibilities: Either there was a switch or not. It allows us to compute the growth probability for each run-length. On the one hand, the belief of an increment of the run-length at the next trial is: (7) where h is the scalar defining the hazard rate. On the other hand, it also allows to express the change-point probability as: (8) with B such that . Note that and thus . Knowing this probability strength and the previous value of the prediction, we can therefore make a prediction for our belief of the probability bias at the next trial t+ 1, prior to the observation of a new datum and resume the prediction / update cycle (see Eqs 4, 7 and 8).

Integrated in our cycle, we update beliefs on all nodes by computing the likelihood of the current datum knowing the current belief at each node, that is, based on observations from trials 0 to t − 1. A major algorithmic difference with the BCP model [58], is that here, the observed data is a Bernoulli trial and not a Gaussian random variable. The random variable is the probability bias used to generate the sequence of events . We will infer it for all different hypotheses on r^t, that is, knowing there was a sequence of r^t Bernoulli trials with a fixed probability bias in that epoch. Such an hypothesis will allow us to compute the distribution by a simple parameterization. Mathematically, a belief on the random variable is represented by the conjugate probability distribution of the binomial distribution, that is, by the beta-distribution . It is parameterized here by its sufficient statistics, the mean and sample size (see Appendix for our choice of parameterization). First, at the occurrence of a switch (for the node r^t = 0) beliefs are set to prior values (before observing any datum): and . By recurrence, one can show that at any trial t > 0, the sufficient statistics can be updated from the previous trial following: (9) As a consequence, is the sample size corrected by the initial condition, that is, . For the mean, the series defined by is the average at trial t over the r + 1 last samples, which can also be written in a recursive fashion: (10) This updates for each node the sufficient statistics of the probability density function at the current trial.

We can now detail the computation of the likelihood of the current datum with respect to the current beliefs: . This scalar is returned by the binary function which evaluates at each node r the likelihood of the parameters of each node whenever we observe a counterfactual alternative outcome o = 1 or o = 0 (respectively right or left) knowing a mean bias and a sample size . For each outcome, the likelihood of observing an occurrence of o, is the probability of a binomial random variable knowing an updated probability bias of , a number p ⋅ r + o of trials going to the right and a number (1 − p) ⋅ r + 1 − o of trials to the left. After some algebra, this defines the likelihood as: (11) with Z such that . The full derivation of this function is detailed in Appendix. This provides us with the likelihood function and finally the scalar value .

Finally, the agent infers at each trial the belief and parameters at each node and uses the marginal predictive probability (see Eq 4) as a readout. This probability bias is best predicted by its expected value as it is marginalized over all run-lengths: (12) Interestingly, it can be proven that if, instead of updating beliefs with Eqs 7 and 8, we set nodes’ beliefs to the constant vector , then the marginal probability is equal to that obtained with the leaky integrator (see Eq 2). This highlights again that, contrary to the leaky integrator, the BBCP model uses a dynamical model for the estimation of the volatility. Still, as for the latter, there is only one parameter which informs the BBCP model that the probability bias switches on average every τ trials. Moreover, note that the resulting operations (see Eqs 4, 7, 8, 11 and 12) which constitute the BBCP algorithm can be implemented online, that is, only the state at trial t and the new datum are sufficient to predict all probabilities for the next trial. In summary, this prediction/update cycle exactly inverts the binary switching generative model and constitutes the Binary Bayesian Change-Point (BBCP) detection model.

Anticipatory pursuit and explicit ratings

We used the binary switching model model to generate the (pseudo-)random sequence of the target’s directions (the alternation of leftward/rightward trials) as the sequence of observations that were used in both sessions (see Fig 3). In the top panel of Fig 3, we show the actual sequence of binary choices (TD, leftward or rightward) of the Bernoulli trials. In the panel below, we compare the true value of the hidden probability bias x₁ (step-like blue curve), and the median predicted values using the leaky integrator (P_leaky, orange color) and BBCP model (P_BBCP, green color), along with the .05 to .95 quantile range (green shaded area), just as in Fig 2B. In the middle panel of Fig 3, we show the median (with the 0.25 ans 0.75 quantiles) anticipatory pursuit velocity (for details see Materials and methods) for the 12 participants, throughout a trial block of 200 trials of the experimental sequence. First, one can observe a trend in the polarity of anticipatory pursuit velocity to be negative for probability bias values below .5 and positive for values above .5. Comparing the raw anticipatory pursuit results with the BBCP agent predictions, it appears qualitatively that both traces evolve in good agreement. In particular, both curves unveil similar delays in detecting and taking into account a switch of the probability bias (while being hidden to the observers), reflecting the time (in the order of a few trials) taken to integrate enough information to build up the estimation of a novel expectation about the probability bias value which parameterizes this Bernoulli trial. In general, results are more variable when the bias is weak (p ≈.5) than when it is strong (close to zero or one), consistent with the well-known dependence of the variance of a Bernoulli trial upon the probability bias (Var(p) = p ⋅ (1 − p)). In addition, the precision (i.e. the inverse of the variance) of the inferred probability bias increases in longer epochs, as information is integrated over more trials. As a result, the inferred probability as a function of time seems qualitatively to constitute a reliable regressor for predicting the amplitude of anticipatory pursuit velocity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Behavioral results, qualitative overview.

For one trial block of 200 trials, we compare the different model-estimated probabilities with respect to the behavioral results. The top row represents the sequence of target directions (TD) that were presented to observers and agents, as generated by the binary switching model (see Fig 1A). We show the evolution of the value of the (true) probability bias P_true (blue line) which is hidden to observers and that is used to generate the TD sequence above. We have overlaid the results of the probability bias predicted with a leaky integrator (P_leaky, orange line) and with the BBCP model (P_BBCP, see Fig 2B, green line). Bottom two rows display the raw behavioral results for the n = 12 observers, by showing their median (lines) and the .25 and .75 quantiles (shaded areas): First, we show the anticipatory pursuit eye velocity, as estimated right before the onset of the visually-driven pursuit. Below, we show the explicit ratings about the expected target direction (or bet scores). These plots show a good qualitative match between the experimental evidence and the BBCP model, in particular after the switches. Note that short pauses occurred every 50 trials (as denoted by vertical black lines, see main text), and we added the assumption in the model that there was a switch at each pause. This is reflected by the reset of the green curve close to the 0.5 level and the increase of the uncertainty after each pause.

https://doi.org/10.1371/journal.pcbi.1007438.g003

In addition, the explicit ratings for the next trial’s expected motion direction (or bet scores, red curve in Fig 3) provided in the other experimental session seem to qualitatively follow the same trend. As with anticipatory pursuit, the series of the participants’ bias guesses exhibits a positive correlation with the true probability bias: The next outcome of will in general be correctly inferred, as compared to a random choice, as reported previously [67]. Indeed, similarly to the amplitude of anticipatory pursuit velocity, we qualitatively compare in Fig 3 the trace of the bet scores with the probability bias inferred by the BBCP model. Moreover, we observe again that a stronger probability bias leads to a lower variability in the bet scores, compared to bias values close to 0.5. Again, a (hidden) switch in the value of the bias is most of the time correctly identified after only a few trials. Finally, note that after every pause (black vertical bar in Fig 3), participants tended to favor unbiased guesses, closer to 0.5. We can speculate that this phenomenon could correspond to a spontaneous resetting mechanism of the internal belief on the probability bias and indeed, we can introduce such an assumption in the model as a reset of the internal belief after each pause. To conclude, the experiment performed in this session shows that the probability bias values that are explicitly estimated by participants are qualitatively similar to the implicit ones which supposedly underlie the generation of graded anticipatory pursuit.

Quantitatively, we now compare the experimental results with the value of the probability bias predicted by the leaky and BBCP algorithms. Compiling results from all participants, we have plotted in Fig 4 the anticipatory pursuit velocity (panel A) and the bet scores (panel B) as a function of the predicted probability biases. In a first analysis, all trials from all participants were pooled together and we show this joint data as an error bar plot as computed for 5 equal partitions of the [0, 1] probability segment showing the median along with the .25 and .75 quantiles. As a comparison, the same method was applied to the true value P_true and to the estimate obtained by the leaky integrator P_leaky. We remind here that the true value of the probability bias was coded at the second layer of the binary switching generative model and is hidden both to the agents and to the human observers. Qualitatively, as we can see in Fig 4A, the predicted probability bias is linearly correlated with the anticipatory pursuit velocity and this dependence is stronger with the the probability bias predicted by the leaky and BBCP algorithms (respectively P_leaky and P_BBCP). In a second analysis, we quantitatively estimated the squared Pearson correlation coefficient and the mutual information between the raw data and the different models, both as computed on the whole data or for each observer individually (see insets in Fig 4). The respective values for the whole dataset (r² = 0.486 and MI = 0.729) and across participants (r² = 0.459 ± 0.104 and MI = 0.707 ± 0.134) are slightly higher than that found by [48] and [11] for anticipatory pursuit measures gathered across experimental trial blocks with fixed direction biases and significantly better than that estimated with the true probability (r² = 0.381 ± 0.083 with p = 0.002 and MI = 0.562 ± 0.107 with p = 0.002) and for that estimated by the leaky-integrator model (r² = 0.366 ± 0.089 with p = 0.002 and MI = 0.622 ± 0.102 with p = 0.004) see inset). Note that p-values were obtained from the Wilcoxon signed-rank test.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Behavioral results, quantitative analysis across participants (n = 12).

To analyze the relation between these behavioral data with the predictions made by models, we first looked at the variability of all these measures conditioned on the predicted probability and gathered over 5 equal partitions of the [0, 1] probability segment. For the 12 participants, we collected an estimate of (A) the amplitude of anticipatory pursuit (aSPEM) and (B) the bet score value. As a regressor, we have used the true probability (, blue color), and the probability bias estimates obtained with a leaky integrator (P_leaky, orange color) and by the BBCP model (P_BBCP, green color). We display these functional relations using an error-bar plot showing the median with .25 and .75 quantiles over the 5 partitions. This shows a monotonous dependency for both behavioral measures with respect to the probability, close to a linear regression, but with different strengths. Second, we summarize in insets quantitative measures of the strength of this dependence for each participant individually, by computing the squared Pearson correlation coefficient r² and the mutual information (MI). Dots correspond to these measures for each individual observer, while the bar gives the median value over the population. This confirms quantitatively that for both experimental measures, there is a strong statistical dependency between the behavioral results and the prediction of the BBCP model, but also that this dependency is significantly stronger than that obtained with the true probability and with the estimates obtained with the leaky integrator (stars denote significative differences, see text for details).

https://doi.org/10.1371/journal.pcbi.1007438.g004

A similar analysis illustrates the relationship between the model-estimated probability bias and the rating value, or bet score, about the expected outcome, which was provided at each trial by participants and is shown in Fig 3. Similarly to the anticipatory pursuit velocity, the rating values are nicely correlated with the probability bias given by the model, as quantified by the squared Pearson correlation coefficient and mutual information across participants (r² = 0.670 ± 0.145 and MI = 1.312 ± 0.364). Importantly, this value is again higher for the BBCP model than for the leaky integrator (r² = 0.551 ± 0.19 with p = 0.018 and MI = 1.117 ± 0.409 with p = 0.028), or with the true probability (r² = 0.490 ± 0.114 with p = 0.002 and MI = 0.940 ± 0.255 with p = 0.002). Further notice that, in order to account for some specific changes observed in the behavioral data after the short pauses occurring every 50 trials, we added the assumption that there was a switch at each pause. However, removing this assumption did not significantly change the conclusions about the match of the model compared to P_true or P_leaky both for eye movements (P_BBCP: r² = 0.452 ± 0.101 and MI = 0.712 ± 0.125, P_leaky: r² = 0.305 ± 0.077 with p = 0.002 and MI = 0.577 ± 0.096 with p = 0.003; P_true: r² = 0.381 ± 0.083 with p = 0.002 and MI = 0.562 ± 0.107 with p = 0.002) and the bet experiment (P_BBCP: r² = 0.652 ± 0.142 and MI = 1.255 ± 0.349, P_leaky: r² = 0.425 ± 0.158 with p = 0.002 and MI = 0.966 ± 0.300 with p = 0.002; P_true: r² = 0.490 ± 0.114 with p = 0.002 and MI = 0.940 ± 0.255 with p = 0.002). To conclude, we deduce that the dynamic estimate of the probability bias produced by the BBCP model is a powerful regressor to explain both the amplitude of anticipatory pursuit velocity and the explicit ratings of human observers experiencing a volatile context for visual motion.

Analyzing inter-individual differences

So far, we have presented the qualitative behavior of individual participants and have reported the quantitative analysis of the data for the fit between experimental and model-inferred estimates of the hidden probability bias. For instance, the experimental measures for the population of 12 participants in Fig 3, support the qualitative match between behavioral data and model predictions, which we then confirmed quantitatively on the whole group of participants. It is important to note that no model fitting procedure was used so far, but only the direct match of the prediction from the BBCP-model resulting from the sequence of binary target directions which were also presented to the human participants, as shown in Fig 2B. Nevertheless, we observed that in both sessions the qualitative match between model and data varied across participants. This was best characterized by differences in the variability of the responses, but also, for instance, by the different characteristic delays after a switch. This reflects the spectrum of individual behavioral choices between exploitation versus exploration [60]. As a consequence, we were interested in characterizing these individual preferences for each individual participant, and potentially to investigate whether this preference co-varied across the two experimental sessions (i.e. across implicit vs explicit response modalities). Crucially, we have seen that the BBCP model is controlled by a single parameter, the hazard rate, or equivalently by its inverse, the characteristic time τ. Also, we have shown that knowing an observed sequence of behavioral responses, we could fit the value of h which would best explain the observations, as quantified by the squared Pearson correlation coefficient or by the mutual information. Thus, by extracting the best-fit parameters for each participant and experimental session, we expect to better understand the variety of inter-individual differences.

Hence, we have fitted the sequence of behavioral responses generated by each participant and for each experimental session, with the predicted probability bias predicted by the BBCP model with different values of its only free parameter, the hazard rate. To avoid any possible bias from the fitting procedure, we tested 1600 linearly spaced values of τ from 1 to 1600 trials. For each, we computed the correlation coefficient with the responses of the BBCP model parameterized by the value of the hazard rate . We then extracted different estimates of h_aSPEM and h_bet, respectively for anticipatory pursuit and the rating scale, by choosing the hazard rate value corresponding to that with maximal correlation coefficient. The scatter plot of the best fit values for each individual is shown in Fig 5. This figure suggests, in the first place, that there is some variability in the best fitted value of the hazard rate in both sessions. Overall, the value of correlation coefficient of the best fit hazard rate was slightly higher than that computed in Fig 3 with r² = 0.471 ± 0.109 for the eye movement session and r² = 0.691 ± 0.152 for the rating scale session. A part of the variability in the estimated hazard rates comes from the limited length of the data blocks, while another part is due to intra-individual and inter-individual variabilities. Overall, the median (with 25% and 75% quantiles) are h_aSPEM = 0.069 (0.065, 0.080) for the anticipatory pursuit session and h_bet = 0.027 (0.012, 0.051) for the rating scale. We observe that these values are close to the (hidden) ground truth value (h = 1/40 = 0.025) used to generate the sequence. In addition, the best-fit hazard rate value is higher for anticipatory pursuit compared to the true value and the rating scale measures. As an interim summary, this analysis reveals that relaxing the free parameter of the BBCP model improves the match of the model to the behavioral data, and that individual best-fitting hazard-rates are variable, especially for the Bet task. Future work might provide important insight about the analysis of these inter-individual differences in terms of each participant’s preference for exploration versus exploitation across different cognitive tasks.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Analysis of inter-individual differences.

(A) We analyzed the behavior of the n = 12 participants individually, by searching for each participant the best value of the model’s single free parameter, the hazard rate h. Estimates were performed independently on both experiments, such that we extracted different estimates of h_aSPEM and h_bet respectively for the anticipatory pursuit velocity and the rating value. The dots correspond to independent estimates of the hazard rate for each individual participant are shown as dots, while the radius is proportional to the squared Person’s correlation coefficient. This plot shows that best fit hazard rates have a median value of and . The values are in general higher than the ground truth (blue line), and in general higher for eye movements (below the diagonal). Note that the dispersion of hazard-rate best-fit estimates is narrower for the eye movement session than for the bet experiment. Such an analysis may suggest that participants ultimately have different mechanisms at the implicit (anticipatory pursuit) and explicit (ratings) levels for guiding their tendency of exploitation versus exploration. (B) To illustrate the models corresponding to these best-fitted values of the hazard rates, we show the predicted probability to the same sequence of TDs, with the lowest (), optimal () and highest () hazard rates (respectively from top to bottom).

https://doi.org/10.1371/journal.pcbi.1007438.g005

The distribution of best-fitted values for each individual participant seemed to qualitatively cluster, but the dataset is still insufficiently large to support the significance of such observation at a quantitative level. Moreover, there is a difference in the distribution of observed hazard rates in both experiments. Indeed, we observed that the marginal distribution for each session is different, with the distribution in the anticipatory pursuit session being narrower than that observed for the rating scale session. In particular, we also observed the same behavior for each trial block independently, suggesting that the origin of this variability mainly comes from inter-individual variability. Second, there is an apparent lack of correlation between the explicit and the implicit estimates of the hazard rate, yet we would need more empirical evidence to prove that this originates from the experimental setup or rather by separate processing of volatility. Such an analysis would suggest that even though the predictive processes at work in both sessions may reflect a common origin for the evaluation of volatility, this estimation is then more strongly modulated by individual preferences when a more explicit cognitive process is at stake.

Measuring adaptation to volatile environments

The time-varying statistical regularities that characterize the environment are likely to influence several cognitive functions. In this study, we have made the choice to focus on a simple and probably mostly unconscious motor behavior (anticipatory pursuit), as well as on the explicit rating of expectation for the forthcoming motion direction. In contrast, we have not addressed the question of whether and how statistical learning affects visual motion perception throughout our model-generated volatile sequences. In an empirical context similar to ours, Maus et al [10] have recently shown that perceptual adaptation for speed estimation occurs concurrently to priming-based anticipatory pursuit throughout a sequence of motion tracking trials with randomly varying speed. They actually found a robust repulsive adaptation effect, with perceptual judgements biased in favor of faster percepts after seeing stimuli that were slower and vice-versa. Concurrently, these authors also found a positive effect on anticipatory pursuit, with faster anticipation after faster stimuli, somehow in agreement with the adaptive properties of anticipatory pursuit that we report here. Moreover, they quantified the trial-history effects on anticipatory pursuit and speed perception by fitting a fixed-size memory model similar to our forgetful agent. They found that anticipatory pursuit and speed perception change over different time scales, with the priming effects being maximized for short-term stimulus history (around 2 trials) and adaptation for longer stimulus history, around 15 trials. Their main conclusion was that perceptual adaptation and oculomotor priming are the result of two distinct readout processes using the same internal representation of motion regularities. Note that both these history lengths can be considered short in comparison to the several hundreds of trials that are commonly used in psychophysics and sensorimotor adaptation studies and that, similar to the present study, the inferred characteristic times are even shorter for the buildup of anticipatory eye movements. However, it is also important to note that in the study by Maus et al [10], the generative model underlying the random sequence of motion trials was different and much simpler than in the present study: In particular the role of environmental volatility was not directly addressed there. This makes a direct comparison between their results and ours difficult beyond a qualitative level.

In spite of a multitude of existing studies investigating the dynamics of sequential effects on visual perception (see for example [5, 7]), only few of them have directly addressed the role of the environmental volatility on the different behavioral outcomes. Meyniel et al [24] have compared the predictions of different models, featuring a dynamic adaptation to the environment’s volatility (equivalent to our forgetful agent model) versus a fixed belief model, on five sets of previously acquired data, including reaction time, explicit reports and neurophysiological measures. Interestingly, they concluded that the estimation of a time-varying transition probability matrix constitutes a core building block of sequence knowledge in the brain, which then applies to a variety of sensory modalities and experimental situations. Consequently, sequential effects in binary sequences would be better explained by learning and updating transition probabilities compared to the absolute item frequencies (as in the present work) or the frequencies of their alternations. The critical difference lies in the content of what is learned (transition probabilities versus item frequencies) in an attempt to capture human behavior. Rather than on transition probabilities, here we focused on the analysis and modeling of human behavior as a function of the frequency of presentation (and its fluctuations in time) of a given event in a binary sequence of alternating visual motion direction. We can speculate that different statistics can play different roles depending on the context, but altogether the study by Meyniel et al [24] and the present one converge to highlight the importance of a dynamic estimate of the hierarchical statistical properties of the environment for efficient behavior. There are also other limits to the agent that we have defined. In this study, we assume that data is provided as a sequence of discrete steps. A similar approach using a Poisson point process allows to extend our model to the continuous time domain, such as addressed by Radillo et al [68]: In their experiments, the authors analyzed the licking behavior of rats in a dynamic environment. The generalization to the time-continuous case is beyond the scope of our current protocol, but it would consist in a natural extension of it to more complex and ecological settings.

Our results demonstrate that the BCCP model is relatively good in mimicking the adaptive changes of both (implicit) anticipatory eye movements and (explicit) ratings of direction expectation in a volatile context. However, these two different behavioral measures, the implicit and the explicit one, are not correlated across individuals. This observation is certainly worth deeper investigation in the future as it raises doubts on the existence of a unique hierarchical system for probabilistic inference. The distinction between implicit and explicit processes in the adaptation to a volatile environment has also been addressed by previous work, especially in the field of statistical learning for language processing (see for example [69, 70]). More related to the present study, Wu et al [71] compared a classical economic decision task with a motor decision task: they found that participants were more risk seeking in the motor task compared to the first one. In addition, Souto et al [72] have recently reported a lack of correlation between the rate of oculomotor adaptation to unexpected jumps of the visual target and the perceptual uncertainty estimated through an explicit jump direction-discrimination experiment. Finally, the degree of explicitness of the information provided to the participants may also play a role in the context of probabilistic learning. In a task similar to ours, where the behavioral choice was not specifically associated to a reward schedule, Santos and Kowler [51] found large similarities but also some differences in the anticipatory eye movements depending on how the information about the probability bias was conveyed, namely through the simple presentation of a biased sequence versus an explicit probability-cueing procedure. In the first condition, the authors reported a weak non-linearity in the dependence of anticipatory pursuit upon the probability of motion direction, yielding an overweight of the extreme values of probability. In contrast, an opposite non-linearity (underweight) was observed when the target direction was visually-cued with a given probability of validity. Note that in our data, we have not found consistent evidence suggesting a clear non-linearity in either sense. Further work is needed to disentangle the possible specificities (e.g. non linearities, also broadly reported in the economic literature, such as a generic aversion to risk [73]) and the general inter-trial and inter-individual correlations across different tasks and different experimental measures of the cognitive adaptation to the environmental volatility.

Hierarchical Bayesian inference in the brain

When we perceive the physical world, make a decision or take an action to interact with it, our brain must deal with an ubiquitous property of it, uncertainty. Uncertainty can arise at different levels and be structured around different characteristic time scales. The theoretical framework of Bayesian probabilistic inference, which provides a formal account for the role of uncertainty at multiple levels, has become very popular as a benchmark of optimal behavior in perceptual, sensorimotor and cognitive tasks [74] and, more generally, as a unified framework for studying the brain [75]. Importantly, plausible hypotheses about the implementation of Bayesian computations —or approximations of them— in the activity of neuronal populations have been proposed [76–78]. However, one should be careful when evaluating the quality of fit of Bayesian inference models for behavioral data, and avoid any over-interpretation of the results. This kind of model fitting aims at evaluating the adequacy of a specific generative inferential model, not of the probabilistic calculus in its detailed implementation. Still, there is actually a common confusion around the idea of a “Bayesian brain”, and we believe that the challenge here is not to validate the hypothesis that the brain implements or not the Bayes’ theorem, or a more complex hierarchical combinations of inferential computations, but rather to test hypotheses about the different generative models that agents may use.

The way expectations act on cognitive processes in general has been investigated in a wide range of domains such as predictive coding [79], active inference [75], motor control [80] and reinforcement learning [11, 60, 65]. Non-stationary observations can also explain why both local and global effects emerge and why local effects persist in the long run even within purely random sequences [28, 81]. This constant update of a general belief on the world can be a consequence of the constant attempt to learn the non-stationary structure of the environment that can change at unpredictable times [81]. Many studies have actually already pointed out the brain’s ability to apprehend non-stationary states in the environment [67, 82]. The relatively strong correlation between model predictions and data that we have found in this study is surprising at a first sight as the epochs with constant probability bias (between two switches) have random lengths, and participants have to adapt to such a volatile environment. However, adaptivity to a volatile environment is one of the most exquisite human skills: When faced with some new observations, the observer has to constantly adapt his/her response to either exploit this information by considering that this observation belongs to the same context of the previous observations, or to explore a novel hypothesis about the context. This compromise is one of the crucial components that we wished to explore and which is well captured by the BBCP model. In particular, the model predicts different aspects of the experimental results, from the variability as a function of the inferred probability, to the dynamics of the behavior following a (hidden) switch. Future work will be needed to address the amplitude and dynamics of modulations of visual perception and other cognitive functions in a model-based volatile environment like the one we formally defined in this study, and to compare them to other implicit and explicit behavioral measures (like anticipatory eye movements and explicit expectation ratings).

The great interest of understanding why and how humans adapt to the fluctuations of the hierarchical probabilistic context is further highlighted by the fact that such adaptivity may deviate in some pathological disorders, such as schizophrenia [4, 83], or across the natural variability of autistic traits [84]. While it was not our original objective, we have analyzed in this study the individual best-fit parameters (hazard rates) of the BCCP model: despite a consistent variability of such parameters across trial blocks of the experiment, we highlighted some noteworthy tendencies for participants to cluster around specific properties of the dynamic adaptation to a volatile probabilistic environment. Most important, this analysis corroborates and strengthens some recent attempts to realize a computational phenotyping of human participants. However, more extensive studies should be conducted to be able to quantitatively titrate inter-individual tendencies and possibly their relation to traits of personality.

Conclusions

• We have developed a Bayesian model of an agent estimating the probability bias of a volatile environment with changing points (switches), such that the agent may decide to stay on the current hypothesis about the environment, or to go for a novel one. This allows to dynamically infer the probability bias across time and directly compare model predictions and experimental data, such as measures of adaptive human behavior.
• We applied such a framework to the case of a probability bias in a visual motion task where we manipulated the target direction probability. We observed a good match between anticipatory smooth eye movements and the predictions of the model, replicating previous findings and providing a novel solid theoretical framework for them [11, 48, 51].
• We also found a good match between model predictions and the explicit rating of the expected target motion direction, a novel result suggesting that this model captures some of the brain computations underlying expectancy based motion prediction, at different cognitive levels.
• Finally, we found that the experimental data of each different participant matched to different types of belief about the volatile environment, some being more or less conservative than others. Interestingly, each of the two experiments (anticipatory eye movements and explicit rating) provided different distributions, opening the perspective for future computational phenotyping using such a volatile setting.

Participants, visual stimuli and experimental design

Twelve observers (29 years old ±5.15, 7 female) with normal or corrected-to-normal vision took part in these experiments. They gave their informed consent and the experiments had received ethical approval from the Aix-Marseille Ethics Committee (approval 2014-12-3-05), in accordance with the declaration of Helsinki.

Visual stimuli were generated using PsychoPy 1.85.2 [85] on a Mac running OS 10.6.8 and displayed on a 22” Samsung SyncMaster 2233 monitor with 1680 × 1050 pixels resolution at 100 Hz refresh rate. Experimental routines were also written using PsychoPy and controlled the stimulus display (see Fig 1). Observers sat 57 cm from the screen in a dark room.

The moving target used in our experiments was a white ring (0.35° outer diameter and 0.27° inner diameter) with a luminance of 102 cd/m2 that moved horizontally on a grey background (luminance 42 cd/m²). Each trial started with a central fixation point displayed for a random duration drawn from a uniform distribution ranging between 400 and 800 ms. Then a fixed-duration 300 ms gap occurred between the offset of the fixation point and the onset of the moving target. The target was then presented slightly offset from the fixation location (step-ramp paradigm [86]), either to the right or to the left, and immediately started moving horizontally toward the center at a constant speed of 15°/s, for 1000 ms. The probability p of rightward motion trials was a time-varying random variable which was constant within an epoch of the sequence of a given random size (see main text for the description of the generative model).

The paradigm included two experimental sessions performed on two distinct (in general consecutive) days by each participant. The two sessions involved the presentation of the same sequence of trials, while collecting a different behavioral response: explicit rating judgments in the first session (the bet experiment), and eye movement recordings in the second session. Asked after the experiment, no observer noticed that the same pseudo-random sequence of target directions was used in both experiments.

Eye movements experiment

Eye movements were recorded continuously with an eye tracking system (Eyelink 1000, SR Research Ltd., sampled at 1000 Hz), using the Python module Pylink 0.1.0 provided by PsychoPy. Horizontal and vertical eye position data were transferred, stored, and analyzed offline using programs written using Jupyter notebooks. The data analyses were implemented using the Python libraries numpy, pandas and pylab. All the scripts for data analysis, as well as for stimulus presentation, data collection, and preparation of figures are available at https://github.com/chloepasturel/AnticipatorySPEM. To minimize measurement errors, the participant’s head movements were restrained using a chin and forehead rest, so that the eyes in primary gaze position were directed towards the center of the screen. In order to enforce accuracy in gaze position and tracking, we implemented an automatic procedure of fixation control. If the distance between the gaze position and the central fixation point during the fixation epoch exceeded 2° of visual angles, the fixation point started flickering and the counter for the fixation duration was reset to 0.

The recorded horizontal and vertical raw gaze position data were numerically differentiated to obtain velocity measures. We adopted an automatic conjoint acceleration and velocity threshold method (the default saccade detection implemented by SR Research) to detect ocular saccades. Saccades and eye-blinks were excluded from eye velocity traces (and replaced by Not-a-Number values in the numerical arrays) before trial averaging and data fitting for the extraction of the oculomotor parameters of interest. In order to extract the relevant parameters of the oculomotor responses, we developed new tools based on a best-fitting procedure of predefined oculomotor patterns and in particular the typical smooth pursuit velocity profile that was recorded in our experiment. A piecewise-defined function was fitted to the different phases of the eye velocity traces: a constant function during fixation, a ramp-like linear function during smooth pursuit anticipation, an increasing sigmoid function during the initiation of visually-guided smooth pursuit, reaching its saturating value during the pursuit steady-state. This analysis was applied to each trial individually and it allowed in particular to estimate the velocity of anticipatory pursuit as the best-fit value of the modeled eye velocity at the moment where the visually-guided pursuit is initiated. Note that this method for estimating anticipatory velocity led to qualitatively identical results to the estimation of the mean eye velocity within an arbitrary temporal window of anticipation, a more classical method that we implemented for instance in a previous study [11]. Some trials were excluded from the analysis as the proportion of missing data-points, due to eye blinks or saccades was considered too large, namely when the missing data exceeded 45 ms during the gap or one third of the total target motion epoch (4.36% of all trials). In addition, trials were also excluded when the eye-movement fitting procedure did not converge, after visual inspection, to a satisfactory match with the data (3.25% of all trials). The python scripts used to analyze eye movements are available at https://github.com/invibe/ANEMO.

The Bet experiment

The aim of the Bet experiment was to collect data related to the individual explicit estimates of the probability for the next outcome of a target motion direction. At the beginning of each trial, before the presentation of the moving target, participants had to answer to the question “How sure are you that the target will go left or right”. This was performed by adjusting a cursor on the screen using the mouse (see Fig 1C). The cursor could be placed at any point along a horizontal segment representing a linear rating scale with three ticks labeled as “Left”, “Right” (at the extreme left and right end of the segment respectively), and “Unsure” in the middle. Participants had to validate their choice by clicking on the mouse left-button and the actual target motion was shown thereafter. The rationale to collect rating responses on a continuous scale instead of a simple binary prediction (Right/Left) was to be able to infer the individual estimate of the direction bias at the single trial scale (in analogy to the continuous interval for the anticipatory pursuit velocity). We called this experiment the “Bet” experiment, as participants were explicitly encouraged to make reasonable rating estimates, just like if they had to bet money on the next trial outcome. Every 50 trials, a “score” was displayed on the screen, corresponding to the proportion of correct direction predictions (Right or Left of the “Unsure” tick) and weighted by the confidence attributed to each answer (the distance of the cursor from the center).