Overview of the PPM-Decay model

This section provides a short overview of the PPM-Decay model. A full exposition is provided at the end of the paper.

The PPM-Decay model is a direct extension of the PPM model [17] incorporating the interpolated smoothing technique of [43] alongside a custom memory retrieval function. PPM is an incremental sequence prediction model: it progresses through sequences symbol by symbol, generating a predictive distribution for the next symbol before it arrives, with this predictive distribution derived from statistics learned from the model’s previous observations. The predictive distribution comprises a list of probabilities for all symbols in the alphabet, with likely continuations receiving high probabilities and unlikely continuations receiving low probabilities.

PPM’s predictions are produced by combining predictions from several sub-models, specifically n-gram models. An n-gram is a sequence of n adjacent symbols; for example, ‘ABCABC’ is an example of a 6-gram. An n-gram model generates predictions by counting and comparing observations of n-grams. To predict what happens after the sequence ‘ABCAB’, a 6-gram model would look at all 6-gram occurrences of the form ‘ABCABX’, where ‘X’ is allowed to vary. The predicted probability of observing ‘C’ having just observed ‘ABCAB’ would then be equal to the number of observations for ‘ABCABC’ divided by the total number of observations for n-grams of the form ‘ABCABX’.

The Markov order of an n-gram model is equal to the length of its n-grams minus one. Different orders of n-gram model have different strengths and weaknesses. Low-order n-gram models are quick to learn, but can only capture simple structure; high-order n-gram models can capture more complex structure, but require more training data. The power of PPM comes from flexibly switching between different Markov orders based on context; this is why it is termed a ‘variable-order’ Markov model.

Different PPM variants use different techniques for combining n-gram models. The original PPM model used a technique called backoff smoothing, where the model chooses one top-level n-gram model from which to generate predictions for a given context, and only calls upon lower-level n-gram models in the case of symbols that have never been seen before by the top-level n-gram model. Here we instead use the interpolated smoothing technique of [43], where lower-order models contribute to predictions for all the symbols in the alphabet. This approach is more computationally expensive, but results in significantly improved predictive power.

The different n-gram models are weighted according to training data availability for the current predictive context: with limited training data, low-order models receive the most weight, but as training data accumulates, high-order models receive increasing weight. The procedure for deriving these weights is termed the ‘escape method’; though various improved escape methods have been introduced over the years, here we retain the escape method from the original PPM model [17], because it generalizes most naturally to the memory-decay procedures considered here.

The PPM-Decay model adds a custom memory retrieval function to PPM. This retrieval function modifies how n-grams are counted: whereas in the original PPM model, each n-gram observation contributes a fixed amount to the respective n-gram count, in the PPM-Decay model the contribution of a given n-gram observation decreases as a function of the time elapsed since the observation. This function is termed the decay kernel; an example decay kernel is plotted in Fig 1. In the following experiments we explore how decay kernels may be constructed that help predictive performance for particular kinds of sequences, and that reflect the dynamics of human auditory memory.

Experiment 1: Memory decay helps predict sequences with changing statistical structure

The original PPM algorithm weights all historic observations equally when predicting the next symbol in a sequence. This represents an implicit assertion that all historic observations are equally representative of the sequence’s underlying statistical model. However, if the sequence’s underlying statistical model changes over time, then older observations will be less representative of the current statistical model than more recent observations. In such scenarios, an ideal observer should allocate more weight to recent observations than historic observations when predicting the next symbol.

Various weighting strategies can be envisaged representing different inductive biases about the evolution of the sequence’s underlying statistical model. A useful starting point is an exponential weighting strategy, whereby an observation’s salience decreases by a constant fraction every time step. Such a strategy is biologically plausible in that the system does not need to store a separate trace for each historic observation, but instead can simply maintain one trace for each statistical regularity being monitored (e.g. one trace per n-gram), which is incremented each time the statistical regularity is observed and decremented automatically over time. This exponential-weighting strategy can also be rationalised as an approximation to optimal Bayesian weighting for certain types of sequence structures [44].

We will now describe a proof-of-concept experiment to demonstrate the intuitive notion that such weighting strategies can improve predictive performance in the PPM algorithm. This experiment used artificial symbolic sequences generated from an alphabet of five symbols, where the underlying statistical model at any particular point in time was defined by a first-order Markov chain. A first-order Markov chain defines the probability of observing each possible symbol conditioned on the immediately preceding symbol; second-order Markov chains are Markov chains that take into account two preceding symbols, whereas zeroth-order Markov chains take into account zero preceding symbols. Our sequence-generation models were designed as hybrids between zeroth-order and first-order Markov chains, reflecting PPM’s capacity to model sequential structure at different Markov orders. These generative models took the form of first-order Markov chains, constructed as follows. First, we sampled five first-order conditional distributions from a symmetric Dirichlet prior with concentration parameter 0.1, with each distribution corresponding to a different conditioning symbol from the alphabet. Second, we averaged these conditional distributions with a 0th-order distribution sampled from the same Dirichlet prior. The resulting Markov chain models can be represented as two-dimensional transition matrices, where the cell in the ith row and the jth column identifies the probability of observing symbol j given that the previous symbol was i (Fig 2A). Zeroth-order structure is then manifested as correlations between transition probabilities in the same column, and can be summarised in marginal bar plots (Fig 2A).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Illustrative plots for Experiment 1.

A) Example sequence-generation models as randomly generated in Experiment 1. The bar plots describe 0th-order symbol distributions, whereas the matrices describe 1st-order transition probabilities. B) Repeated-measures plot indicating how predictive accuracy for individual sequences (N = 500, hollow circles) increases after the introduction of an exponential-decay kernel. C) Absolute changes in predictive accuracy for individual sequences, as summarised by a kernel density estimator. The median accuracy change is marked with a solid vertical line.

https://doi.org/10.1371/journal.pcbi.1008304.g002

Each sequence began according to an underlying statistical model constructed by the above procedure, with the first symbol in each sequence being sampled from the model’s stationary distribution. At the next symbol, the underlying statistical model was either preserved with probability .99 or discarded and regenerated with probability .01. The new symbol was then sampled from the resulting statistical model conditioned on the immediately preceding symbol. This procedure was repeated to generate a sequence totalling 500 symbols in length.

Individual experimental trials were then conducted as follows. The PPM-Decay model was presented with one symbol at a time from a sequence constructed according to the procedure defined above, and instructed to return a predictive probability distribution for the next symbol. A single prediction was then extracted from this probability distribution, corresponding to the symbol assigned the highest probability. Prediction success was then operationalized as the proportion of observed symbols that were predicted correctly.

This experimental paradigm was used to evaluate a PPM-Decay model constructed with an exponential-decay kernel and a Markov order bound of one. This kernel is parametrized by a single half-life parameter, defined as the time interval for an observation’s weight to decrease by 50%. This half-life parameter was optimized by evaluating the model on 500 experimental trials generated by the procedure described above, maximizing mean prediction success over all trials using Rowan’s [45] Subplex algorithm as implemented in the NLopt package [46], and refreshing the model’s memory between each trial. The resulting half-life parameter was 12.26. The PPM-Decay model was then evaluated with this parameter on a new dataset of 500 experimental trials and compared with an analogous PPM model without the decay kernel.

The results are plotted in Fig 2B and 2C. They indicate that the exponential-decay kernel reliably improves the model’s performance, with the median percentage accuracy increasing from 48.8% to 62.2%. The exponential-decay kernel causes the algorithm to downweight historic observations, which are less likely to be representative of the current sequence statistics, thereby helping the algorithm to develop an effective model of the current sequence statistics and hence generate accurate predictions. Correspondingly, we can say that the exponential-decay model better resembles an ideal observer than the original PPM model.

Experiment 2: Memory decay helps predict musical sequences

We now consider a more complex task domain: chord sequences in Western music. In particular, we imagine a listener who begins with zero knowledge of a musical style, but incrementally acquires such knowledge through the course of musical exposure, and uses this knowledge to predict successive chords in chord sequences. This process of musical prediction is thought to be integral to the aesthetic experience of music, and so it is of great interest to music theorists and psychologists to understand how these predictions are generated [16, 47–50].

Chord sequences in Western music resemble sentences in natural language in the sense that they can be modeled as sets of symbols drawn from a finite dictionary and arranged in serial order. Such chord sequences provide the structural foundation of most Western music. For the purpose of modeling with the PPM algorithm, it is useful to translate these chord sequences into sequences of integers, which we do here using the mapping scheme described in Methods. For example, the first eight chords of the Bob Seger song ‘Think of Me’ might be represented as the integer sequence ‘213, 159, 33, 159, 213, 159, 33, 159’.

Here we consider chord sequences drawn from three musical corpora: a corpus of popular music sampled from the Billboard ‘Hot 100’ charts between 1958 and 1991 [51], a corpus of jazz standards sampled from an online forum for jazz musicians [52], and a corpus of 370 chorale harmonizations by J. S. Bach [53], translated into chord sequences using the chord labeling algorithm of [54] (see Methods for details). These three corpora may be taken as rough approximations of three musical styles: popular music, jazz music, and Bach chorale harmonizations. While we expect these three corpora each to be broadly consistent with general principles of Western tonal harmony [55], we also expect each corpus to possess distinctive statistical regularities that differentiate the harmonic languages of the three musical styles [52, 56–58]. Fig 3 displays example chord sequences from these three corpora, alongside their corresponding integer encodings.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Sample chord sequences analyzed in Experiment 2.

A) represents the popular music corpus (‘Night Moves’, by Bob Seger), B) represents the jazz corpus (‘Thanks for the Memory’, by Leo Robin), and C) represents the Bach chorale harmonization corpus (‘Mit Fried und Freud ich fahr dahin’, by J. S. Bach). Each chord is labeled by its integer encoding within the chord alphabet for the respective corpus. Each chord sequence corresponds to the first eight chords of the first composition in the downsampled corpus. Each chord is defined by a combination of a bass pitch class (lower stave) and a collection of non-bass pitch classes (upper stave). For visualization purposes, bass pitch classes are assigned to the octave below middle C, and non-bass pitch classes to the octave above middle C.

https://doi.org/10.1371/journal.pcbi.1008304.g003

We expect the underlying sequence statistics to vary as we progress through a musical corpus. Sequence statistics are likely to change significantly at the boundaries between compositions, but they are also likely to change within compositions, as the chord sequences modulate to different musical keys, and travel through different musical sections. Similar to Experiment 1, we might therefore hypothesize that some kind of decay kernel should help the listener maintain an up-to-date model of the sequence statistics, and thereby improve predictive performance.

However, unlike Experiment 1, the chord sequences within a given musical corpus are likely to share certain statistical regularities. If the corpus is representative of a given musical style, then these statistical regularities will correspond to a notion of ‘harmonic syntax’, the underlying grammar that defines the harmonic conventions of that musical style. An ideal model will presumably take advantage of these stylistic conventions. However, the exponential-decay kernel from Experiment 1 is not well-suited to this task, because observations from historic sequences continuously decay in weight until they make essentially no contribution to the model. This is not ideal because these historic sequences will still contribute useful information about the musical style. Here we therefore evaluate a modified exponential-decay kernel, where memory traces decay not to zero but to a positive asymptote (see e.g. Fig 1). Such a kernel should provide a useful compromise between following the statistics of the current musical passage and capturing long-term knowledge of a style’s harmonic syntax.

We conducted our experiment as follows. For each musical corpus, we simulated a listener attempting to develop familiarity with the musical style by listening to one chord sequence every day, corresponding to one composition randomly selected from the corpus without repetition, for 100 days. We supposed that the listener began each chord sequence at the same time of day, so that the beginning of each successive chord sequence would be separated by 24-hour intervals, and we supposed that each chord in each chord sequence lasted one second in duration. Similar to Experiment 1, we supposed that the listener constantly tried to predict the next chord in the chord sequence, but this time we operationalized predictive success using the cross-entropy error metric, defined as the mean negative log probability of each chord symbol as predicted by the model. This metric is more appropriate than mean success rate for domains with large alphabet sizes, such as harmony, because it assigns partial credit when the model predicts the continuation with high but non-maximal probability. A similar metric, perplexity, is commonly used in natural language processing (cross-entropy is equal to the logarithm of perplexity). We used the cross-entropy metric to evaluate two decay kernels: the exponential-decay kernel evaluated in Experiment 1, termed the ‘Decay only’ kernel, and a new exponential-decay kernel incorporating a positive asymptote, termed the ‘Decay + long-term learning’ model. We found optimal parametrizations for these kernels using the same optimizer as Experiment 1 [45], and compared the predictive performance of the resulting optimized models to a standard PPM model without a decay kernel. Each model was implemented with a Markov order bound of four, which seems to be a reasonable upper limit for the kinds of Markovian regularities present in Western tonal harmony [49, 59, 60].

Fig 4 describes the performance of these two decay kernels. Examining the results for the three different datasets, we see that the utility of different decay parameters depends on the musical style. For the popular music corpus, incorporating exponential decay improves the model’s performance by c. 1.9 bits, indicating that individual compositions carry salient short-term regularities that the model can better leverage by downweighting historic observations. Introducing a non-zero asymptote to the decay kernel does not improve predictive performance on this dataset, indicating that long-term syntactic regularities contribute very little to predictive performance over and above these short-term regularities in popular music. A different pattern is observed for the jazz and Bach chorale corpora, however. In both cases, the decay-only model performs no better than the original PPM model, presumably because any improvement in capturing local statistics is penalized by a corresponding deterioration in long-term syntactic learning. However, incorporating a non-zero asymptote in the decay kernel allows the model both to upweight local statistics and still achieve long-term syntactic learning, thereby improving predictive performance by c. 1.5 bits.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Predictive performances for different decay kernels in Experiment 2.

Each composition contributed one cross-entropy value for each decay kernel; these cross-entropy values are expressed relative to the cross-entropy values of the original PPM model, and then summarised using kernel density estimators. Median performance improvements are marked with solid vertical lines.

https://doi.org/10.1371/journal.pcbi.1008304.g004

These analyses have two main implications. First, they show that more advanced decay kernels are useful for producing a predictive model that better approximates ideal performance in the cognitive task of harmony prediction. The nature of these improved kernels can be related directly to the statistical structure of Western music, where compositions within a given musical style tend to be characterized by local statistical regularities, yet also share common statistical structure with other pieces in the musical style. An ideal-observer model of harmony prediction ought therefore to recognize these different kinds of statistical structure. Second, these analyses offer quantitative high-level insights into the statistical characteristics of the three musical styles. In particular, the popular music analyses found that long-term learning offered no improvement over a simple exponential-decay kernel, implying that the harmonic structure of popular music is dominated by local repetition. In contrast, both the jazz analyses and the Bach chorale analyses found that both exponential decay and long-term learning were necessary to improve from baseline performance, implying that chord progressions in these styles reflect both short-term statistics and long-term syntax to significant degrees.

Experiment 3: Memory decay helps to explain the dynamics of auditory pattern detection

The PPM model has recently been used to simulate how humans detect recurring patterns in fast auditory sequences [3]. Barascud et al. used an experimental paradigm where participants were played fast tone sequences derived from a finite pool of tones, with the sequences organised into two sections: a random section (labelled ‘RAND’) and a regular section (labelled ‘REG’). The random section was constructed by randomly sampling tones from the frequency pool, whereas the regular section constituted several repetitions of a fixed pattern of frequencies from the pool. These two-stage sequences, termed ‘RAND-REG’ sequences, were contrasted with ‘RAND’ sequences which solely comprised one random section. The participant’s task was to detect transitions from random to regular sections as quickly as possible (see Fig 5 for an example trial).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Example analysis of a single trial in Experiment 3.

The three panels plot each tone’s frequency, change-point statistic, and information content respectively. ‘Nominal transition’ denotes the point at which the pattern changes from random tones to a repeating pattern of length 10. This repetition starts to become discernible after 10 tones (‘Effective transition’), at which point the sequence becomes fully deterministic. Correspondingly, information content (or ‘surprise’) drops, and triggers change-point detection at ‘Detection of transition’.

https://doi.org/10.1371/journal.pcbi.1008304.g005

Two simplified sequence types were also included for the purpose of baselining reaction times: ‘CONT’ sequences, which were simplified versions of RAND sequences that comprised just one repeated tone of a given frequency, and ‘STEP’ sequences, which were simplified versions of RAND-REG sequences where all tones within a given section had the same frequency, with this frequency differing between sections. We used the STEP trials to estimate baseline response times, and normalized the RAND-REG response times by subtracting these baseline response times (see Methods for more details).

These experimental stimuli were constructed according to a well-defined statistical process, and it would be straightforward to derive a model that achieves provably optimal performance on the task given a well-defined performance metric. However, Barascud et al. reasoned that the cognitive mechanisms underlying fast auditory pattern recognition would be unlikely to be tailored to exact repetition, because exact repetitions are uncommon in naturalistic auditory environments. Instead, they supposed that human performance would be better characterized by more generic regularity detection mechanisms, such as those embodied in the PPM algorithm.

In particular, [3] suggested that listeners maintain an internal predictive model of incoming tone sequences that is incrementally updated throughout each sequence, and that listeners monitor the moment-to-moment surprise experienced by this model. They modeled this process using PPM as the predictive model, and operationalized surprise as the information content of each tone, defined as the tone’s negative log probability conditioned on the portion of the sequence heard so far. The authors proposed that listeners detect section changes based on the evolution of information content throughout the stimulus; in particular, changes from random to regular precipitate a sharp drop in information content, reflecting the transition from unpredictability to predictability.

Examining information content profiles produced by the PPM model, [3] concluded that an ideal observer should detect the transition from random to regular sections by the fourth tone of the second occurrence of the regular tone cycle. Analyzing behavioral and neuroimaging data, the authors found that participants reached this benchmark when the cycle length was small (5, 10 tones) but not when it was large (15, 20 tones). In other words, the ideal-observer model replicated human performance well for short cycle lengths, but some kind of cognitive constraints seemed to impair human performance for large cycle lengths.

One candidate explanation for this impaired performance is the limited capacity of auditory short-term memory. In order to detect a cycle repetition, the listener must compare incoming tones to tones that occurred at least one cycle ago. To achieve this, the listener’s auditory short-term memory must therefore span at least one cycle length. Short cycles may fit comfortably in the listener’s short-term memory, thereby supporting near-optimal task performance, but longer cycles may progressively test the limits of the listener’s memory capacity, resulting in progressively worsened performance.

An important question is whether this memory capacity is determined by temporal limits or informational limits. A temporal memory limit would correspond to a fixed time duration, within which events are recalled with high precision, and outside of which recall performance suffers. Analogously, an informational limit would correspond to a fixed number of tones that can be recalled with high fidelity from short-term memory, with attempts to store larger numbers of tones resulting in performance detriment.

Both kinds of capacity limits have been identified for various stages of auditory memory. Auditory sensory memory, or echoic memory, is typically characterized by its limited temporal capacity but high informational capacity. Auditory working memory has a more limited informational capacity, and a temporal capacity that can be extended for long periods through active rehearsal. Auditory long-term memory seems to be effectively unlimited in both temporal and informational capacity [23–25, 61].

The auditory sequences studied by Barascud et al. used very short cycle lengths, of the order of 1 s, and were therefore likely to fall within the bounds of echoic memory. Temporal limits to echoic memory are well-documented in the literature: echoic memory seems to persist for a few seconds, and is susceptible to interference from subsequent sounds [62]. Informational limits to echoic memory are less well understood. The most relevant precedent to this paradigm in the literature seems to be [63], studying change detection in sequences of pure tones where total sequence duration ranged from 60 to 2000 ms. In this regime, change-detection performance was largely unaffected by sequence duration, and was instead constrained by the number of tones in the sequence, pointing to an informational limit to echoic memory. However, related work has studied change detection where the tones were presented simultaneously rather than sequentially, and failed to find compelling evidence for memory capacity limits over and above sensory processing limits [64, 65]. Another relevant precedent is [66], who played participants pairs of adjacent click trains each of 1 s duration, and asked participants to answer whether the second click train was a repetition of the first click train. Performance decreased as click rate increased, which could reflect capacity limits in echoic memory, but could also reflect capacity limits in sensory processing. In summary, informational limits to echoic memory have received limited characterization in the literature, but nonetheless seem equally plausible to temporal limits as explanations of Barascud et al.’s results [3].

We conducted a behavioral experiment to test these competing explanations. We based this experiment on the regularity detection task from Barascud et al., and created six experimental conditions that orthogonalised two stimulus features: the number of tones in the cycle (10 tones or 20 tones), and the temporal duration of each tone in the cycle (25 ms, 50 ms, or 75 ms). We reasoned that if performance were constrained by informational capacity, then it would be best predicted by the number of tones in the cycle, whereas if performance were constrained by temporal limits, it would be best predicted by the total duration of each cycle. We were particularly interested in the pair of conditions with equal cycle duration but different numbers of tones per cycle (10 × 50 ms = 20 × 25 ms); a decrease in performance in the latter condition would be evidence for informational constraints on regularity detection.

The behavioral results are summarized in Fig 6. Response accuracies are plotted in Fig 6A in terms of the sensitivity metric from signal detection theory. Response accuracy was close to ceiling performance across all conditions, with the exception of the condition with the maximum-duration cycles (20 tones each of length 75 ms), where some participants fell away from ceiling performance. This is consistent with [3]; the task manipulations change how long it takes for the participant to detect the change, but they mostly do not prevent the participant from eventually detecting the change. Following [3], we therefore focus on interpreting reaction-time metrics (Fig 6B) rather than response accuracy. Here we see a clear effect of the number of tones in the cycle, with 10-tone cycles eliciting considerably lower reaction times than 20-tone cycles. This is consistent with the notion of an informational capacity to echoic memory. In particular, comparing the two conditions with equal cycle duration but different numbers of tones per cycle (10 × 50 ms tones; 20 × 25 ms tones), we see that increasing the number of tones substantially impaired performance even when cycle duration stayed constant.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Behavioral results for Experiment 3.

A) Participant d-prime scores by condition, as summarized by violin plots and Tukey box plots. B) Participant mean response times by condition, as summarized by violin plots and Tukey box plots. C) As B, except benchmarking response times against the 25 ms conditions.

https://doi.org/10.1371/journal.pcbi.1008304.g006

Fig 6B does not show a clear effect of tone duration. However, the figure does not account for the repeated-measures structure of the data, meaning that between-condition effects may be partly masked by individual differences between participants. To achieve a more sensitive analysis, Fig 6C takes advantage of the repeated-measures structure of the data, and plots each participant’s response time in the 50-ms and 75-ms conditions relative to their response time in the relevant 25-ms condition. Here we again see null or limited effects of tone duration, except in the case of the maximum-duration condition (20 tones each of length 75 ms), where reaction times seem higher than in the corresponding 25-ms and 50-ms conditions. We tested the reliability of this effect by computing each participant’s difference in mean response time between the 25-ms and 75-ms conditions for the 20-tone cycles, and subtracting the analogous difference in response times for the 10-tone cycles, in other words: {RT(75 ms, 20 tones) − RT(25 ms, 20 tones)} − {RT(75 ms, 10 tones) − RT (25 ms, 10 tones)}. This number summarizes the extent to which increasing tone duration has a stronger effect on reaction times for cycles containing more tones. Using the bias-corrected and accelerated bootstrap [67], the 95% confidence interval for this parameter was found to be [2.08, 5.93]. The lack of overlap with zero indicates that the effect was fairly reliable: increasing tone duration from 25-ms and 75-ms had a stronger negative effect on reaction times for 20-tone cycles than for 10-tone cycles. This interaction between tone duration and cycle length was also evident from a 3 x 2 repeated-measures ANOVA (F(2, 44) = 8.76, η² = .07, p < .001).

To summarize, then: the behavioral data indicate that performance in this regularity-detection task was primarily constrained by the number of tones in the repeating cycles, rather than their duration. However, the data do suggest a subtle negative effect of tone duration which may manifest for cycles containing large numbers of tones.

We now consider how these effects may be reproduced by incorporating memory effects into the PPM model. Instead of the decay kernel solely operating as a function of time, as in Experiments 1 and 2, it must now account for the number of tones that have been observed by the listener. Various such decay kernels are possible. Here we decided to base our decay kernel on the following psychological ideas, inspired by previous research into echoic memory [23, 24, 68]:

1. Echoic memory operates as a continuously updating buffer that stores recent auditory information.
2. While a memory remains in the buffer, it is represented with high fidelity, and is therefore a reliable source of information for regularity detection mechanisms.
3. The buffer has a limited temporal and informational capacity. Memories will remain in the buffer either until a certain time period has elapsed, or until a certain number of subsequent events has been observed.
4. Once a memory leaves the buffer, it is represented in a secondary memory store.
5. Observations in this secondary memory store contribute less strongly to auditory pattern detection, and gradually decay in salience over time, as in Experiments 1 and 2.

These principles, formalized computationally and applied to the continuous tone sequences from the behavioral experiment, result in the decay kernels described in Fig 7. In each case the buffer is limited to a capacity of 15 tones, which corresponds to a time duration of 0.375 s for 25-ms tones, 0.75 s for 50-ms tones, and 1.125 s for 75-ms tones. While the n-gram observation remains within this buffer, its weight is w₀ = 1.0; once the memory exits the buffer its weight drops to w₁ = 0.6, and thereafter decays exponentially to w_∞ = 0 with a half life of t_0.5 = 3.5 s. The precise parameters of this decay kernel come from manual optimization to the behavioral data, and it should be noted that these parameters may well be task-dependent; nonetheless, we will show that each qualitative component of the decay kernel seems to be necessary to explain the observed pattern of results.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Decay kernels employed in Experiment 3.

The temporal duration of the buffer corresponds to the buffer’s informational capacity (15 tones) multiplied by the tone duration.

https://doi.org/10.1371/journal.pcbi.1008304.g007

Weight decay by itself is not sufficient to cause memory loss, because PPM computes its predictions using ratios of event counts, which are preserved under multiplicative weight decay. We therefore introduce stochastic noise to the memory retrieval component of the PPM model, meaning that weight decay reduces the signal-to-noise ratio, and thereby gradually eliminates the memory trace of the original observation. In our optimized model this noise component is implemented by sampling from a Gaussian with zero mean and standard deviation σ_ϵ = 0.8, taking the absolute value, and adding this to the total weight over all observations for a given n-gram, this total weight being an arbitrarily large non-negative number. We avoid negative noise samples so as to avoid negative event counts, for which the PPM model is not well-defined.

Applied to an individual trial, the model returns the information content for each tone in the sequence, corresponding to the surprisingness of that tone in the context of the prior portion of the sequence (Fig 5). Following [3], we suppose that the listener identifies the transition from random to regular tone patterns by detecting the ensuing drop in information content. We model this process using a non-parametric change-detection algorithm that sequentially applies the Mann-Whitney test to identify changes in a time series’ location while controlling the false positive rate to 1 in 10,000 observations [69].

All stimuli were statistically independent from one another, and so responses should not be materially affected by experiences on previous trials. For simplicity and computational efficiency, we therefore left the PPM-Decay model’s long-term learning weight (w_∞) fixed at zero, and reset the model’s memory store between each trial.

We analyzed 6 different PPM-Decay configurations, aiming to understand how the model’s different features contribute to task performance, and which are unnecessary for explaining the perceptual data. Specifically, we built the proposed model step-by-step from the original PPM model, first adding exponential decay, then adding retrieval noise, then adding the memory buffer. We tested three versions of the final model with different buffer capacities: 5 items, 10 items, and 15 items. We manually optimized each model configuration to align mean participant response times to mean model response times, producing the parameter sets listed in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Optimized model parameters for Experiment 3.

https://doi.org/10.1371/journal.pcbi.1008304.t001

Original PPM.

As expected, the original PPM model proved not to be sensitive to tone length or to alphabet size (Fig 8A). Furthermore, the model systematically outperformed the participants, with an average reaction time of 6.23 tones compared to the participants’ mean reaction time of 12.90.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Modeling participant data in Experiment 3.

Participant data (mean response times) are plotted as white circles, whereas different model configurations (mean simulated response times) are plotted as solid bars. Error bars denote 95% confidence intervals computed using the central limit theorem. A) Progressively adding exponential weight decay and retrieval noise to the original PPM model. B) Progressively adding longer buffers to the PPM-Decay model.

https://doi.org/10.1371/journal.pcbi.1008304.g008

Ethics statement

The present research was approved by the research ethics committee of University College London (Project ID Number: 1490/009). Written informed consent was obtained from each participant.

Model

Our PPM-Decay model embodies a predictive processing account of auditory regularity detection. It supposes that listeners acquire an internal model of incoming sounds through automatic processes of statistical learning, and use this model to generate prospective predictions for upcoming auditory events. The model derives from the PPM algorithm [17, 43], but adds three psychological principles:

1. a) The memory salience of a given observation decays as a function of the timepoints of subsequently observed events and the timepoint of memory retrieval.
2. b) There exists some noise, or uncertainty, in memory retrieval.
3. c) A limited-capacity memory buffer constrains learning and prediction. Contiguous events (n-grams) must fit into this buffer to be internalized or to contribute to prediction generation.

Each of these three features can be enabled or disabled in isolation. In ideal-observer analyses, such as Experiments 1 and 2, it often makes sense to omit features b) and c), because they correspond to cognitive constraints that typically impair prediction. Here we therefore omit these two features for the ideal-observer analyses (Experiments 1 and 2), but retain them for the behavioral analyses in Experiment 3.

Many variants of PPM exist in the literature [17, 40, 43, 109]. Our formulation incorporates the interpolated smoothing technique of [43], but avoids techniques such as Kneser-Ney smoothing, exclusion, update exclusion, and state selection, because it is not obvious how to generalize these techniques to decay-based models where n-gram counts can fall between zero and one, and because we do not yet have compelling evidence that human cognition employs analogous methods. Nonetheless, an interesting topic for future work is to explore the cognitive relevance and potential computational implementations of these techniques.

Learning.

The model learns by counting occurrences of different sequences of length n termed n-grams (), where n is termed the n-gram order. As in PPM, the model counts n-grams for all n ≤ n_max (), where n_max is the n-gram order bound. A three-symbol sequence (e₁, e₂, e₃) contains six n-grams: (e₁), (e₂), (e₃), (e₁, e₂), (e₂, e₃), and (e₁, e₂, e₃).

We suppose that n-grams are extracted from a finite-capacity buffer (Fig 9). Successive symbols enter and leave this buffer in a first-in first-out arrangement, so that the buffer represents a sliding window over the input sequence. The buffer has two capacity limitations: itemwise capacity and temporal capacity. The itemwise capacity, n_b, determines the maximum number of symbols stored by the buffer; the temporal capacity, t_b, determines the maximum amount of time that a given symbol can remain in the buffer before expiry. Generally speaking, itemwise capacity will be the limiting factor at fast presentation rates, whereas temporal capacity will be the limiting factor at slow presentation rates. As n-grams may only be extracted if they fit completely within the buffer, these capacities bound the order of extracted n-grams. Correspondingly, we constrain n_max (the n-gram order bound) not to exceed n_b (the itemwise buffer capacity).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Schematic figure of accumulating observations within a memory buffer.

Weights for the n-gram “AB” are displayed as a function of time, assuming an itemwise buffer capacity (n_b) of 5, a buffer weight (w₀) of 1.5, an initial post-buffer weight (w₁) of 1, a half life (t_0.5) of 1 second, and an asymptotic post-buffer weight (w_∞) of 0.

https://doi.org/10.1371/journal.pcbi.1008304.g009

In PPM, n-gram observations are recorded by incrementing a counter. Our PPM-Decay model also stores the ordinal position within the input sequence when the observation occurred; this is necessary for simulating the temporal dynamics of auditory memory. For each n-gram x, we define count(x) as the total number of observations of x, and pos(x) as a list of ordinal positions in the input sequence when these observations occurred, defined with respect to the final symbol in the n-gram. pos(x) is initialized as an empty list; each time a new n-gram x is observed, the respective ordinal position is appended to the list. count(x) is then represented implicitly as the length of pos(x).

The input sequence is processed one symbol at a time, from beginning to end. Observing the ith symbol, e_i, yields up to n_max n-gram observations, corresponding to all n-grams in the buffer that terminate with the most recent symbol: . If the buffer component of the model is enabled, an n-gram observation will only be recorded if it fits completely within the itemwise and temporal capacities of the buffer; the former constraint is ensured by the constraint that n_max ≤ n_b, but the latter must be checked by comparing the current timepoint (corresponding to the final symbol in the n-gram) with the timepoint of the first symbol of the n-gram. If the current ordinal position is written pos_end, and the n-gram length is written size(x), then the necessary and sufficient condition for n-gram storage is where τ_i is the ith timepoint in the input sequence, and t_b is the temporal buffer capacity, as before. Table 2 describes the information potentially learned from training on the sequence (a, b, a).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. n-grams learned from training on the sequence a, b, a.

https://doi.org/10.1371/journal.pcbi.1008304.t002

Memory decay.

In the original PPM algorithm, the influence of a given n-gram observation is not affected by the passage of time or the encoding of subsequent observations. This contrasts with the way in which human observers preferentially weight recent observations over historic observations [26, 27, 29–31, 44, 110]. This inability to capture recency effects limits the validity of PPM as a cognitive model.

Here we address this problem. We suppose that the influence, or weight, of a given n-gram observation varies as a function both of the current timepoint and the timepoints of the symbols that have since been observed. This weight decay function represents the following hypotheses about auditory memory:

1. Each n-gram observation begins in the memory buffer (Fig 9). Within this buffer, observations do not experience weight decay.
2. Upon leaving the buffer, observations enter a secondary memory store. This transition is accompanied by an immediate drop in weight.
3. While in the secondary memory store, observations experience continuous weight decay over time, potentially to a non-zero asymptote.

These hypotheses must be considered tentative, given the scarcity of empirical evidence directly relating memory constraints to auditory prediction. However, the notion of a short-lived memory buffer is consistent with pre-existing concepts of auditory sensory memory [23–25], and the continuous-decay phenomenon is consistent with well-established recency effects in statistical learning [26, 27, 29–31, 44, 110].

We formalize these ideas as follows. For readability, we write pos(x, i) for the ith element of pos(x), corresponding to the ordinal position of the ith observation of n-gram x within the input sequence, defined with respect to the final symbol of the n-gram. Similarly, we write time(x, i) as an abbreviation of τ_{pos(x, i)}, the timepoint of the ith observation of n-gram x. We then define w(x, i, t) as the weight for the ith observation of n-gram x for an observer situated at time t: Here w₀ is the buffer weight, w₁ is the initial post-buffer weight, and w_∞ is the asymptotic post-buffer weight (w₀ ≥ w₁ ≥ w_∞ ≥ 0). The function f defines an exponential decay with half-life equal to t_0.5, with t_0.5 > 0: time_expire(x, i) denotes the timepoint at which the ith observation of n-gram x expires from the buffer, computed as the earliest point when either the temporal capacity or the itemwise capacity expires. The temporal capacity expires when t_b seconds have elapsed since the first symbol in the n-gram, whereas the itemwise capacity expires when n_b symbols have been observed since the first symbol in the n-gram: An illustrative memory-decay profile is shown in Fig 10.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Illustrative weight decay profile.

This figure plots the weight of an n-gram of length one as a function of relative observer position, assuming that new symbols continue to be presented every 0.05 seconds. Model parameters are set to t_b = 2, n_b = 15, w₀ = 1.0, t_0.5 = 3.5, w₁ = 0.6, and w_∞ = 0, as optimized in Experiment 3.

https://doi.org/10.1371/journal.pcbi.1008304.g010

Memory traces accumulate over repeated observations of the same n-gram. We define W(x, t), the accumulated weight for an n-gram x, as

As currently specified, memory decay does not necessarily cause forgetting, because the same information may be preserved in the ratios of n-gram weights even as the absolute values of the weights shrink. For example, consider a pair of n-grams AB and AC with weights 4 and 1 respectively, both undergoing exponential decay to an asymptotic weight of 0. From these n-gram weights, the model can estimate the probability that B follows A as p(B|A) = 4/(4 + 1) = 0.8. After one half-life, the new counts are 2 and 0.5 respectively, but the maximum-likelihood estimate remains unchanged: p(B|A) = 2/(2 + 0.5) = 0.8.

A better account of forgetting can be achieved by supposing that memory traces must compete with noise factors introduced by imperfections in auditory memory; in this case, shrinking the absolute values of n-gram weights decreases their signal-to-noise ratio and hence induces forgetting. Here we model imperfections in memory retrieval by adding truncated Gaussian noise to the retrieved weights: (1) where W*(x, t) is the retrieved weight of n-gram x at time t, and represents Gaussian noise uncorrelated across n-grams or timepoints. Setting to zero disables the noise component of the model.

Prediction.

Traditionally, a maximum-likelihood n-gram model estimates the probability of symbol e_i given context by taking all n-grams beginning with and finding the proportion that continued with e_i. For n ≤ i: where denotes an n-gram probability estimator of order n, is the number of times n-gram occurred in the training set, and denotes the concatenation of sequence and symbol x. The n-gram model predicts from the previous n − 1 symbols, and therefore constitutes an (n − 1)th-order Markov model. Note that the estimator defaults to a uniform distribution if , when the context has never been seen before. Note also that the predictive context of a 1-gram model is the empty sequence .

To incorporate memory decay into a maximum-likelihood n-gram model, we replace the count function c with the retrieval weight function W*. For n ≤ i: This decay-based model degenerates to the original maximum-likelihood model when w₀ = 1, t_b → ∞, n_b → ∞, σ_ϵ = 0 (i.e. an infinite-length memory buffer with unit weight and no retrieval noise).

High-order n-gram models take into account more context when generating their predictions, and are hence capable of greater predictive power; however, this comes at the expense of greater tendency to overfit to training data. Conversely, low-order models are more robust to overfitting, but this comes at the expense of lower structural specificity. Smoothing techniques combine the benefits of both high-order and low-order models by merging n-gram models of different orders, with model weights varying according to the amount of training data. Here we use interpolated smoothing as introduced by [43, 104]. For n ≤ i, the unnormalized interpolated n-gram estimator is recursively defined as a weighted sum of the nth-order maximum-likelihood estimator and the (n − 1)th-order interpolated estimator: (2) where is the nth-order unnormalized interpolated n-gram estimator, is the nth-order maximum-likelihood estimator, is the alphabet size, and a_n is a function of the context sequence that determines how much weight to assign to , the maximum-likelihood n-gram estimator of order n.

The unnormalized interpolated estimator defines an improper probability distribution that does not necessarily sum to 1. We therefore define as the normalized interpolated estimator: Note that the need for normalization can alternatively be avoided by redefining for n = 0 in Eq (2), meaning that the interpolated smoothing terminates with a proper probability distribution. However, we keep the original definition to preserve equivalence with [43] and [18].

The weighting function a_n corresponds to the so-called “escape mechanism” of the original PPM algorithm. [111] review five different escape mechanisms, termed “A” [17], “B” [17], “C” [40], “D” [112], and “AX” [113] [43, 104], each corresponding to different weighting functions a_n. Of these, “C” tends to perform the best in data compression benchmarks [111]. However, methods “B”, “C”, “D”, and “AX” do not generalize naturally to decay-based models; in particular, it is difficult to ensure that the influence of an observation is a continuous function of its retrieved weight w*. We therefore adopt mechanism “A”.

In its original formulation, mechanism “A” gives the higher-order model a weight of a_n = 1 − 1/(1 + T_n), where T_n is the number of times the predictive context has been seen before (which can be interpreted as the observer’s familiarity with the preceding sequence of n − 1 tokens). When the context has never been seen before, T_n = 0 and a_n = 0, and the estimator relies fully on the lowest-order model; as T_n → ∞, a_n → 1, and the estimator relies fully on the highest-order model. In the original PPM algorithm, the number of times that the predictive context has been seen before is equal to the sum of the weights (or counts) for each possible continuation: Introducing memory-decay reduces the weights for these prior observations, decreasing the model’s effective experience, and preferentially weighting lower-order models, as might be expected. However, retrieval noise is problematic, because it positively biases the retrieved weights (see Eq (1)), causing the algorithm to overestimate its familiarity with its predictive context, and to overweight high-order predictive contexts as a result. We compensate for this by subtracting the expected value of the retrieval noise’s contribution to T_n, which can be computed from standard results for the truncated normal distribution as , and truncating at zero: Putting this together, we have (for i ≥ n):

For its final output, the model selects the maximum-order available normalized interpolated estimator. The available orders are constrained by three factors:

1. The n-gram order bound: the model cannot predict using n-grams larger than n_max.
2. The sequence: the predictive context must fit within the observed sequence.
3. The buffer: the predictive context must fit within the buffer at the point when the incoming symbol is observed.

Putting this together, the selected n-gram order for generating predictions from a context of becomes: The final model output is then:

The n-gram order bound, n_max, constrains the length of n-grams that are learned by the model. However, it is often more convenient to speak in terms of the model’s Markov order, m_max, defined as the number of preceding symbols that contribute towards prediction generation. A single n-gram model generates predictions with a Markov order of n − 1; correspondingly, m_max = n_max − 1.

Fig 11 illustrates the interpolated smoothing mechanism. Here we imagine that a model with a Markov order bound of two processes the sequence “abracadabra”, one letter at a time, and then tries to predict the next symbol. The highest-order interpolated distribution, at a Markov order of two, is created by averaging the order-2 maximum-likelihood distribution with the order-1 interpolated distribution, which is itself created by averaging the order-1 maximum-likelihood distribution with the order-0 interpolated distribution. The resulting interpolated distribution combines information from maximum-likelihood models at every order.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Illustration of the interpolated smoothing mechanism.

This smoothing mechanism blends together maximum-likelihood n-gram models of different orders. Here the Markov order bound is two, the predictive context is “abracadabra”, and the task is to predict the next symbol. Columns are identified by Markov order; rows are organized into weight distributions, maximum-likelihood distributions, and interpolated distributions. Maximum-likelihood distributions are created by normalizing the corresponding weight distributions. Interpolated distributions are created by recursively combining the current maximum-likelihood distribution with the next-lowest-order interpolated distribution. The labelled arrows give the weight of each distribution, as computed using escape method “A”. The “Order = −1” column identifies the termination of the interpolated smoothing, and does not literally mean a Markov order of −1.

https://doi.org/10.1371/journal.pcbi.1008304.g011

We have implemented the resulting model in a freely available R package, “ppm”, the core of which is written in C++ for speed. With this package, it is possible to define a PPM-Decay model customized by the eight hyperparameters summarized in Table 3. The package also supports simpler versions of PPM-Decay, where (for example) the buffer functionality is disabled but the exponential-decay functionality is preserved. The resulting models can then be evaluated on arbitrary symbolic sequences. The package may be accessed from its open-source repository at https://github.com/pmcharrison/ppm or its permananent archive at https://doi.org/10.5281/zenodo.2620414.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Summary of PPM-Decay hyperparameters.

https://doi.org/10.1371/journal.pcbi.1008304.t003

Musical corpora

Bach chorale corpus.

This corpus was derived from the ‘371 chorales’ dataset from the KernScores repository [53]. This dataset comprises four-part chorale harmonizations by J. S. Bach, as collected by his son C. P. E. Bach and his student Kirnberger, and eventually digitally encoded by Craig Sapp. The 150th chorale harmonization is omitted from Sapp’s dataset as it is not in four parts, leaving 370 chorales in total. This dataset uses the **kern representation scheme [115], designed to convey the core semantic information of traditional Western music notation. For example, the following text represents the first two bars of the chorale harmonization ‘Mit Fried und Freud ich fahr dahin’:

4D  4F  4A  4d

=1  =1  =1  =1

4C# 4A  4e  4a

4D  4d  4f  4a

4E  4B  4e  4g

8F#L    8AL 8dL 4dd

8G#J    8BJ 8eJ .

=2  =2  =2  =2

4A  8cnXL   8eL 4ccnX

.   8dJ 8f#J    .

4E  8eL 4g# 4b

.   8dJ .   .

4AA;    4c; 4e; 4a;

4E  [4c 4g  4cc

=3  =3  =3  =3

We derived chord sequences from these **kern representations by applying the harmonic analysis algorithm of [54], which selects from a dictionary of candidate chords using a template-matching procedure. Here we used an extended version of this template dictionary, described in Table 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. The dictionary of chord templates used in constructing the Bach chorale corpus.

https://doi.org/10.1371/journal.pcbi.1008304.t004

We computed one chord for each quarter-note beat, reflecting the standard harmonic rhythm of the Bach chorale style, and collapsed consecutive repetitions of the same chord into one chord, as before. Fig 3C shows the result for the first eight chords of the chorale harmonization ‘Mit Fried und Freud ich fahr dahin’.

Behavioral experiment

Modeling reaction time data

We modeled participants’ reaction times using the new PPM-Decay model presented in Model. We modeled each trial separately, resetting the model’s memory after each trial.

We modeled participants’ change detection processes using a non-parametric change-detection algorithm that sequentially applies the Mann-Whitney test to identify changes in a time series’ location while controlling the false positive rate [69, 116]. We used the algorithm as implemented in the ‘cpm’ R package [116], setting the desired false positive rate to one in 10,000, and the algorithm’s warm-up period to 20 tones.

For comparison with the participant data, we computed representative model reaction times for each condition by taking the mean reaction time over all trials where the model successfully detected a transition, excluding any trials where the model reported a transition before the effective transition (this resulted in excluding 0.32% of trials). We used R and C++ for our data analyses [117]; our PPM-Decay implementation is available at https://github.com/pmcharrison/ppm and https://doi.org/10.5281/zenodo.2620414. Raw data, analysis code, and generated outputs are archived at https://doi.org/10.5281/zenodo.3603058.