The Auditory Pitch-Motion Hypothesis

Here, we use the term auditory pitch-motion to refer to motion of an auditory stimulus in frequency space. We note that previous studies have used the term auditory motion. However, we prefer to refer to auditory pitch-motion in order to avoid potential confusions with motion of an auditory stimulus in physical space. Auditory pitch motion conveys important information to listeners in both music and speech contexts [4], [12], [15]–[18]. For example, in music, the notes of a melody change in frequency in systematic ways to convey information about tonality [17] and contribute to perceived accent structure [19]. In the speech domain, patterns of rises and falls in pitch over time (an element of speech prosody) help determine word boundaries, distinguish between questions and statements, disambiguate semantic content, place emphasis, and convey information about the emotional state of the speaker [15], [16], [18]. Of interest here is the general hypothesis that auditory pitch motion provides listeners a framework from which to predict the future time-course of a stimulus, that is, ‘what’ will happen ‘when’.

The auditory pitch-motion hypothesis proposes that listeners are sensitive to the perceived pitch velocity of an auditory stimulus [14], where pitch velocity, V, is approximated by Δf/Δt. Throughout this article, we will use Δf to refer to the change in frequency in units of Hertz (Hz) between two time points, namely the onset and offset of a stimulus, while Δt refers to the amount of time that has elapsed between the same two time points and as such is equivalent to the duration of the stimulus.

Two critical tenets of the auditory pitch-motion hypothesis are that (1) listeners assume that auditory stimuli will tend to maintain a relatively constant pitch velocity and (2) listeners use this information to generate expectations about the future timing and pitch characteristics of an unfolding stimulus. As such, an unexpected increase in the pitch distance covered per unit time is predicted to lead to the perceived expansion of the corresponding time interval in order to equalize perceived velocity across the stimulus. Conversely, an unexpected decrease in the duration of a stimulus interval is predicted to lead to a subjective shrinking of perceived pitch distance of the same interval, again so that the stimulus conforms to the expectation of constant velocity. Moreover, an unexpected decrease in the frequency change across an interval should lead to a subjective shrinking of perceived duration, while an unexpected increase in frequency change is predicted to lead to a subjective expansion of the corresponding duration.

In this study, we tested these two tenets of the auditory pitch-motion hypothesis by having listeners judge either the perceived time change (i.e., duration) or perceived pitch change of a comparison stimulus relative to the perceived duration or pitch change of a referent (standard) stimulus. Both the standard and comparison stimuli were frequency glides that changed linearly in frequency over their temporal extent at a constant velocity (Fig. 1). For each listener, the velocity of the standard glide was fixed at one of three values (500 Hz/s, 1000 Hz/s, or 1500 Hz/s) to provide a constant-velocity referent. Based on the auditory pitch-motion hypothesis, listeners’ were assumed to use the velocity of the standard glide to generate predictions about the amount of time change or frequency change that would be expected to occur between the onset and offset of the comparison glide. To either violate or reinforce the constant velocity assumption, the comparison velocity varied from trial to trial and took on values of 500 Hz/s, 1000 Hz/s, and 1500 Hz/s. This meant that in some cases, the standard and comparison velocities matched, and in other cases, they did not.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Schematic of stimuli and task for the current experiment.

Here, only the 1000-Hz velocity condition is illustrated. The time-change (i.e., duration-judgment) task is shown on the top line, and the pitch-change task is shown on the bottom line. On each trial, a constant-velocity standard glide was presented. Then a variable-velocity comparison stimulus was presented; the comparison velocity took on one of three values. Comparison levels were varied parametrically on the to-be-judged dimension; thus values of the ignored dimension were adjusted in order to maintain the prescribed velocity. For the time-change task, listeners indicated whether the comparison was shorter or longer than the standard. For the pitch-change task, listeners indicated whether the comparison changed more or less in pitch than the standard.

https://doi.org/10.1371/journal.pone.0070646.g001

Predictions

Predictions for both tasks were generated by formalizing the auditory pitch-motion hypothesis in terms of an imputed velocity model [10], [11], [20]. In this model, perceived duration and pitch change are represented as a weighted combination of actual and expected values along the to-be-judged dimensions. Expected values for both tasks are derived from the assumption that participants use the constant (referent) velocity, V, established by the standard glide when asked to judge stimulus attributes of the comparison glide. For the time change task, the perceived duration of the comparison glide, τ, is given by:(1)

Here, the actual duration of the comparison glide is represented by Δt, and the expected duration of the comparison glide is represented by Δt_E where Δt_E = Δf/V. For the pitch-change task, the perceived pitch change, φ, of the comparison glide is given by:(2)

Here, the actual frequency change of the comparison glide is represented by Δf and the expected frequency change of the comparison glide is represented by Δf_E, where Δf_E = ΔtV. Equations 1 and 2 [20] reflect the idea that perceived time change or pitch change can be estimated from a weighted combination of actual and expected values, where for both tasks, the expected values along the to-be-judged dimension are derived from the standard equation for velocity, V = Δf/Δt; the form differs simply because the computed expected value (Δf_E or Δt_E) is in the numerator for the pitch-change task, but in the denominator for the time-change task. Nonetheless, for both tasks, perceptions along the to-be-judged dimension are distorted in the direction of the constant velocity established by the standard (referent) glide. The weight value, w ∈ [0,1], represents a free parameter that can be used to estimate the magnitude of these perceptual distortions due to imputed velocity, with smaller values of w corresponding to larger perceptual distortions.

The auditory pitch-motion hypothesis and associated imputed pitch velocity model make the following behavioral predictions for the time-change and pitch-change tasks. In general, the predictions for both tasks are based on the view that perceived duration and pitch change of the comparison glide will reflect both the actual values (Δt or Δf depending on the task-relevant dimension) and the expected values derived from the velocity of the standard glide (Δt_E = Δf/V or Δf_E = ΔtV). Note that for both tasks, when the velocities of the standard and comparison glides are the same, the actual and expected values of the comparison glide along the task-relevant dimension are equal (Δt = Δt_E or Δf_E = Δf) and in this case no systematic perceptual distortions are predicted. However, systematic perceptual distortions are predicted for both tasks with the velocities of the two glides differ.

For the time-change task, when the velocity of the comparison glide is faster than the standard, the comparison changes more in frequency per unit time than expected given the velocity of the standard, and thus the expected duration of the comparison is longer than its actual duration (Δt_E>Δt). In this case, the perceived duration of the comparison is predicted to be overestimated. On the other hand, when the velocity of the comparison glide is slower than the standard, the comparison changes less in frequency per unit time than expected given the velocity of the standard, and thus the expected comparison duration is shorter than actual duration (Δt_E<Δt). In this case, the perceived duration of the comparison is predicted to be underestimated.

Conversely, for the pitch-change task, when the velocity of the comparison glide is faster than the standard, the duration of the comparison is shorter than would be expected given the velocity the standard. Thus the expected value for pitch change is less than the actual value (Δf_E<Δf), and perceived pitch change is predicted to be underestimated. Conversely, when the comparison velocity is slower than the standard, comparison time change is longer than standard time change. Thus the expected value for pitch change is greater than the actual value (Δf_E>Δf), and comparison pitch change is expected to be systematically overestimated.

In sum, the auditory-pitch motion hypothesis and associated imputed velocity model make opposite patterns of predictions for the time-change and pitch-change tasks. When the velocities of the comparison and standard glides match, then no systematic distortions in perceived duration or pitch change are predicted. However when the velocity of the comparison glide is faster than the standard, perceived duration and pitch change are predicted to be over- and underestimated, respectively. In contrast, when the velocity of the comparison glide is slower than the standard, perceived duration and pitch change are predicted to be under- and overestimated, respectively. In the context of the design of the present study, in which we separately manipulated the velocities of the standard and comparison glides, the predictions of the auditory pitch-motion hypothesis translate into the expectation of a three-way interaction between standard velocity, comparison velocity, and task (time-change vs. pitch change).

Ethics Statement

The Institutional Review Board of Michigan State University approved all procedures, which were in accordance with the Declaration of Helsinki. Written informed consent was obtained from all participants prior to the experiment.

Design

The design of the experiment was a 2 (Task: time-change, pitch-change)×3 (Standard Velocity: 500 Hz/s, 1000 Hz/s, 1500 Hz/s)×3 (Comparison Velocity: 500 Hz/s, 1000 Hz/s, 1500 Hz/s)×8 (Comparison Level: −20%, −15%, −10%, −5%, +5%, +10%, +15%, +20%) mixed factorial. Task and Standard Velocity were between-participants factors, while Comparison Velocity and Comparison Level were within-participant factors. Participants judged either the time change (i.e., duration) or pitch change of a variable-velocity comparison stimulus relative to a constant-velocity standard stimulus.

Participants

One hundred and eight individuals (80 female; ages 18–35 years) from Michigan State University participated in return for course credit in an introductory psychology class. Participants self-reported normal hearing and varied in number of years of formal music training (0–15 years, M = 3.8, SD = 3.7). Participants were randomly assigned to one of the three standard velocity conditions (500 Hz/s, n = 36; 1000 Hz/s, n = 40; 1500 Hz/s, n = 32) and completed either the time-change task (n = 56) or the pitch-change task (n = 52).

Stimuli

Stimuli were pure-tone linear frequency glides, ramped linearly over the first and last 5 ms to eliminate acoustic artifacts. Standard glides ascended in pitch at one of three velocities (500 Hz/s, 1000 Hz/s, 1500 Hz/s); for each fixed standard velocity condition, comparison glides also ascended in pitch at one of three velocities (500 Hz/s, 1000 Hz/s, 1500 Hz/s). For the time-change task, the standard glide was always 1000 ms in duration and the comparison glide took on one of 8 values with equal probability (1000 ms ±5%, 10%, 15%, 20%). For the pitch-change task, the change in frequency for the standard glide was always 1000 Hz and the comparison glide took on one of 8 values with equal probability (1000 Hz ±5%, 10%, 15%, 20%). Thus, for all combinations of standard and comparison velocities, participants completing each task experienced the same standard and set of comparisons along the task-relevant dimension. In order for that to be the case, the value of the comparison along the task-irrelevant dimension necessarily varied in order to maintain the prescribed velocity. The starting frequencies of the standard and comparison glides were independently randomized from trial to trial, taking on one of three values (476 Hz, 600 Hz, 756 Hz).

Apparatus

Stimuli were generated using MATLAB software (The Mathworks, Inc.). Stimulus generation and response collection were controlled using E-Prime 2.0.8.73 software (Psychology Software Tools, Inc.) running on Dell Optiplex computers. Auditory stimuli were presented at a comfortable listening level (∼70 dB SPL) over Sennheiser HD 280 Pro headphones. Responses were made using a serial response box; “shorter” vs. “longer” and “more pitch change” vs. “less pitch change” responses were made by pressing left and right buttons, respectively.

Procedure

On each trial, participants heard a pair of glides and judged the time change or pitch change of the comparison (second) stimulus relative to the standard (first). For the time-change task, participants judged whether the comparison was “shorter” or “longer” than the standard, and for the pitch-change task participants judged whether the comparison changed “more” or “less” in pitch than the standard (Fig. 1). Before completing the experiment proper, participants heard recorded instructions and completed an 18-trial training block with corrective feedback. During training, participants heard only comparison levels of ±20%. Participants then completed two experimental blocks with no feedback; a short break was allowed between blocks. In each block, participants provided three responses to each combination of Comparison Velocity and Comparison Level, for a total of 72 trials. Overall, listeners completed a total of 144 trials; thus 6 observations were obtained for each Comparison Velocity×Comparison Level combination. The experiment lasted approximately 30 minutes.

Data Analysis

Proportions of “longer” responses (time-change task) and “more pitch change” responses (pitch-change task) were determined for each participant for each of the eight values of Comparison Level for each Comparison Velocity, averaged over the two test blocks. Relative points of subjective equality (PSEs) for each of the resulting psychometric curves were then estimated for each participant using the following procedure, which fits response proportions to a cumulative normal (Gaussian) probability density function [21]. First, response proportions were converted to z-scores. A correction was applied to proportions of 0 or 1 so that they could be z-transformed; 0 and 1 proportions were first converted to 1/2N or 1–(1/2N), respectively, where N = 6 trials per condition. After z-transformation (which results in psychometric curves becoming approximately linear), we estimated the best-fit straight line through the eight data points making up the psychometric curve using a least-squares approach. From the resulting best-fit linear equation, we estimated the relative point of subjective equality (PSE), which corresponds to the value of the comparison stimulus along the judged dimension which yielding 50% “longer” or “more pitch change” responses, respectively. Note that PSE is typically expressed in units corresponding to the physical stimulus dimension being manipulated (i.e., Hz or ms). However, we relativized our stimulus dimensions and expressed them as percentage changes with respect to the value of the standard stimulus so that relative PSEs would be comparable across tasks. The relative PSE measure quantifies the degree to which perceived duration or pitch change was under- or overestimated relative to the point of objective equality; in signal detection terms, relative PSE is measure of perceptual bias, with negative values indicating overestimation, while positive values indicate underestimation.

Proportions of “longer” and “more pitch change” responses were also fit with the imputed velocity model. For each participant, we simultaneously fit three psychometric functions corresponding to the three comparison velocity conditions. That is, separately for each task and standard velocity, the model simultaneously took into account the eight comparison levels and three comparison velocities; thus w values were estimated on the basis of 24 data points (6 observations per point). Specifically, predicted response proportions were obtained from the following equation:(3)where x is a vector of perceived duration or pitch change at all comparison levels and all comparison velocites estimated from equations 1 and 2 [10]. Model fits estimated the best-fitting value of the weight parameter, w, by minimizing root mean square error between observed response proportions and response proportions predicted by equation 3. Mean w values are reported for the time-change and pitch-change tasks separately for all standard velocity conditions.

Possible Alternative Explanations

We have interpreted the current results as being consistent with the auditory pitch-motion hypothesis. However, in reminder tasks that involve the presentation of a fixed standard on each trial followed by a variable comparison, it has been shown that listeners sometimes ignore the standard altogether, and instead compare the current value of the comparison to a remembered standard [22]. Additionally, when comparison values vary along a continuum, a convergence towards the mean value is often observed such that larger magnitude stimuli tend to be underestimated while smaller magnitude stimuli tend to be overestimated [23]–[25]. Thus, it is important to consider a possible alternative explanation for the current results, which is that listeners ignored the standard stimulus entirely, and the observed perceptual distortions instead resulted from using the average comparison velocity as the constant-velocity referent. Critically, this hypothesis predicts an identical pattern of results across all three Standard Velocity conditions. However, since we observed different patterns of results for the three standard velocities (as indexed by the significant three-way interaction), an explanation based on using an average of comparison velocities as a constant-velocity referent is not viable.

A second potential alternative explanation for the current results concerns the use of the same response button assignment for all listeners in all conditions. That is, “shorter” and “more pitch change” responses were always made using a left-side button, while “longer” and “less pitch change” responses were always made using a right-side button. Previous work has shown stimulus-response compatibility effects for timing [26] and pitch judgments [27]–[29]. That is, responses are faster and more accurate when “faster”/”shorter” responses are made with a left-side button and “slower”/”longer” responses are made with a right-side key. In the pitch domain, an advantage is observed when relatively high and low frequencies are responded to by pressing relatively high and low buttons, respectively, on a vertical button display. These results have been interpreted to reflect a natural mapping of some stimulus dimensions to physical space.

Although a similar effect has not been demonstrated for velocity, it nonetheless seems important to consider whether the results of the present study could be explained by a natural mapping of slow velocities to the left side and fast velocities to the right side. Assuming such a mapping, our results are potentially explainable in part by a bias to press the left button (“shorter”, “more pitch change”) when the comparison velocity was slower than the standard and the right button (“longer”, “less pitch change”) when the comparison velocity was faster than the standard. In order to test this possibility, we replicated the 1000-Hz standard velocity condition from the current experiment with button assignments reversed. The results showed that the stimulus-response mapping does not matter for the tested tasks and thus provide additional support for the auditory pitch-motion hypothesis (see Fig. A and supplemental text in File S1). That is, independent of stimulus-response mapping, duration was overestimated and pitch change underestimated when the comparison velocity was faster than the standard, while duration was underestimated and pitch change overestimated when the comparison velocity was slower than the standard.

Magnitude of Perceptual Distortions as a Function of Overall Velocity

One prediction that is explicitly made by the auditory pitch-motion hypothesis regards the effect of velocity on the strength of predictions made by listeners about future stimulation, and subsequent perceptual distortions. Specifically, the auditory pitch-motion hypothesis predicts a range of rates within which perceptual distortions should obtain, and in particular, that the magnitude of perceptual distortions should increase with increasing velocity [12]. This hypothesis has previously been supported by experiments requiring listeners to judge either the timing or pitch of a target tone embedded in tone sequences [10], [11]. Both the timing and pitch of the target tone were varied, although only one dimension was task relevant, as in the current study. Similar to the current results, listeners’ perception of target timing and pitch were distorted in the direction consistent with listeners generating expectations about time-pitch location of the target based on assumed constant velocity across the tone sequence. Moreover, perceptual distortions increased in magnitude with increasing stimulus velocity. The current results are concordant with these previous findings, in that here, we observed largest w values (and thus smallest perceptual distortions) for the slowest standard velocity condition (500 Hz/s).

Potential Effects of Glide Direction

In the current study, we examined time-change and pitch-change judgments for glides that ascended in frequency, but not for glides that descended in frequency. We restricted ourselves here to studying ascending glides, because we have directly compared ascending and descending auditory stimuli in a number of previous studies [10], [11], [14], and found very little difference between them. For example, in two studies quantifying the degree to which listeners make use of pitch-velocity information to predict the timing and frequency of a single tone in discrete tone sequences [10], [11], we found that the magnitude of perceptual distortions based on pitch velocity were slightly larger for descending than for ascending tone sequences, but the results were qualitatively the same. We have previously proposed an auditory gravity hypothesis to explain these results. The key assumption of this hypothesis is that descending auditory stimuli are heard as accelerating relative to ascending stimuli; thus, from the perspective of the present study, we might expect perceptual distortions based on pitch velocity to be exaggerated for descending glides, but the overall pattern would be expected to be the same.

An alternative hypothesis for descending glides comes from a study by van Wassenhove and colleagues [30], where the subjective duration of ascending glides was overestimated relative to the duration of descending glides. This is consistent with comparisons of auditory stimuli that increase versus decrease in intensity, and thus sound as if they are approaching or receding, respectively [31]–[34]. Specifically, approaching stimuli are consistently judged as longer than receding stimuli. Thus, it is possible that, in the current study, the duration of ascending glides would be in general overestimated relative to the duration of descending glides. We suspect that behavioral differences between ascending and descending glides in the current study would be unlikely, however, for the following reason. In the current study, duration and pitch-change judgments were relative – that is, they involved judging stimulus attributes of a comparison stimulus relative to a standard that had the same direction, here: ascending). Thus, even if there would be an overall difference between ascending and descending glides in terms of perceived duration, we would expect to see the same three-way interaction between the velocities of the two glides and task. Nonetheless, further studies are needed to conclusively tease apart these two possibilities.

Predictive Accounts of Auditory Perception

There are several theoretical perspectives that explicitly posit that human perceptual processes are supported by active prediction. For example, predictive coding accounts assume that perceivers make use of statistical regularities in patterns of stimulation in order to generate predictions about upcoming stimuli [1], [2]. On this view, stimulus statistics derived from previous stimulation are specifically used to generate a prior distribution in the Bayesian sense to which unfolding stimuli can be compared. When predictions are violated, an “error signal” results; in terms of dependent measures, the predictive coding account specifically predicts larger neural responses for unexpected (relative to expected) stimulus events. Evidence for this account comes from a number of sources, including for example, mismatch negativity (MMN) paradigms, where a repeated stimulus regimen is occasionally interrupted by a “deviant”, or unexpected, token. EEG and MEG recordings reveal a “mismatch” response to the deviant item occurring after approximately 100–250 ms [9], [35], which is assumed to reflect the error signal [1], [36]. Similarly, BOLD responses observed in fMRI studies are typically larger for unexpected events than for expected events [37], [38].

An alternative, but not necessarily mutually exclusive, account of predictive elements of perceptual processing is dynamic attending theory, which is primarily an account of how expectations can be built up in time [4], [7], [39]. In brief, dynamic attending theory posits that attention is an inherently rhythmic process; that is, attention waxes and wanes as time passes. However, the attentional rhythm is easily entrained by external stimulation. Through entrainment, attentional peaks come to be aligned with the expected onset times of important stimulus events; thus, events occurring at expected times are better processed than events occurring at unexpected times. Supporting this idea is a volume of behavioral research showing that the perception of the timing or pitch of an event is better when the event occurs at the expected time based on a previous temporal context [6]–[8], [40], [41]. See ref. [42] for a recent neural framework that explains temporal predictions by appealing to neural oscillations.

The auditory pitch-motion hypothesis is consistent with an extension of dynamic attending theory to a two-dimensional (time-pitch) stimulus space [4], [43]. From this perspective, auditory stimuli varying in pitch over time can be conceptualized as tracing a trajectory through time-pitch space. Deviations from the predicted trajectory then lead to perceptual distortions like those observed here. An important avenue for future research will be to explore how such predictions might be specified neurally, in particular from an oscillatory framework.

Conclusion

The current study provides a novel test of an auditory pitch-motion hypothesis whereby listeners’ expectations about the velocity of an auditory stimulus were predicted to yield systematic perceptual distortions. Consistent with this prediction, listeners’ judgments about the task-relevant stimulus dimension of a glide (i.e., time or pitch) were influenced by expectations stemming from the constant velocity of a standard (referent) glide. Most striking was support for the prediction that we would observe an opposite pattern of perceptual distortions for the time-change and pitch-change tasks. Time change (i.e., duration) was overestimated when the comparison velocity was faster than the standard and underestimated when the comparison was relatively slow, while pitch change was underestimated when the comparison was fast and overestimated when the comparison velocity was slower than the standard. Overall, the current findings provide further support for the active, predictive nature of human auditory perceptual processing.