https://orcid.org/0000-0001-8341-9226
Figures
Abstract
When a stimulus is associated with an external reward, its chance of being consolidated into long-term memory is boosted via dopaminergic facilitation of long-term potentiation in the hippocampus. Given that higher temporal distance (TD) has been found to discount the subjective value of a reward, we hypothesized that memory performance associated with a more immediate reward will result in better memory performance. We tested this hypothesis by measuring both behavioral memory performance and brain activation using functional magnetic resonance imaging (fMRI) during memory encoding and retrieval tasks. Contrary to our hypothesis, both behavioral and fMRI results suggest that the TD of rewards might enhance the chance of the associated stimulus being remembered. The fMRI data demonstrate that the lateral prefrontal cortex, which shows encoding-related activation proportional to the TD, is reactivated when searching for regions that show activation proportional to the TD during retrieval. This is not surprising given that this region is not only activated to discriminate between future vs. immediate rewards, it is also a part of the retrieval-success network. These results provide support for the conclusion that the encoding-retrieval overlap provoked as the rewards are more delayed may lead to better memory performance of the items associated with the rewards.
Citation: Yoo J, Min S, Lee S-K, Han S (2021) Neural correlates of episodic memory modulated by temporally delayed rewards. PLoS ONE 16(4): e0249290. https://doi.org/10.1371/journal.pone.0249290
Editor: Kiyoshi Nakahara, Kochi University of Technology, JAPAN
Received: August 3, 2020; Accepted: March 16, 2021; Published: April 7, 2021
Copyright: © 2021 Yoo et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data used in this study are available on Figshare (DOI: 10.6084/m9.figshare.13285928.v1) and also included as a Supporting Information file.
Funding: This research was supported by the Brain Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Science and ICT (2017M3C7A1029485) and partially supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government (NRF-2019R1A2C1007399). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Episodic long-term memory is crucial to our daily lives given that our behavior and decisions are based on past experience. The core of episodic memory formation in the brain is thought to be the medial temporal lobe (MTL), which comprises the hippocampus (HC) and the surrounding gyri [1]. In addition, various studies suggest that successful memory formation does not solely depend on MTL functions but also relies on other cortical regions such as the prefrontal cortex (PFC), the premotor cortex, or the posterior parietal cortex (PPC) [2].
However, not all events benefit from these regions to be transformed into long-term memory. To enhance efficiency in memory storage, salient stimuli–either rewarding or aversive–are better remembered than other neutral items [3]. Upon encountering rewarding stimuli, dopamine neurons in the substantia nigra and the ventral tegmental area (VTA), two regions that produce most of the dopamine in the brain, are activated [4]. Dopamine is known to facilitate consolidation of long-term memory via direct projections of the VTA to the HC [5]. This reward-facilitated memory consolidation is observed 24 hours after the initial learning [6] through late long-term potentiation in the HC [7,8]. Various studies have confirmed the effect of reward-facilitated episodic-memory encoding in the human brain. One study revealed that stimuli that elicit greater activation in the dopaminergic midbrain areas are more likely to be recollected three weeks later [9], while another study reported that a high-value reward preceding incidental encoding facilitates memory formation [10]. In summary, a high-value stimulus has a higher chance of being consolidated into the long-term memory (for a review, see [11]).
The subjective value of a reward has been found to vary according to various contexts such as the amount of effort needed for obtaining the reward, probability associated with the reward, or delay before the reward is delivered. Effort discounting refers to the tendency of choosing a reward option associated with less over more required effort. Activation of the nucleus accumbens was negatively correlated with effort demand [12]. Likewise, with a fixed objective value of a reward, its subjective value decreases if the probability of obtaining it decreases [13]. Temporal discounting of reward refers to the tendency of individuals to devalue the reward if its delivery is delayed [14], often resulting in choosing smaller sooner over larger later rewards. There have been various attempts to capture this phenomenon into mathematical equations, and a model using hyperbolic function has provided the most parsimonious account so far [15,16]. A neuroimaging study has corroborated this by finding that the subjective value of delayed rewards derived from the hyperbolic model is represented in the ventral striatum, the mPFC, and the posterior cingulate cortex [17].
Given that a TD of a reward reduces the activation of dopamine neurons as if a reward of a smaller magnitude has been expected [18–20], one could link this with reward-motivated memory-encoding by questioning whether devalued rewards due to a higher TD act like a small-magnitude reward in memory encoding. With regard to reward contexts other than TD, a study examined effects of reward uncertainty and magnitude on episodic memory and found that only reward outcome, but not reward uncertainty, affects episodic memory [21]. However, to our knowledge, no study attempted to test whether temporally discounted rewards differentially affect episodic memory formation. Therefore, in this study, we aim to investigate whether temporal discounting of reward is reflected in reward-motivated episodic memory encoding, and if so, whether it involves differential HC-VTA activity.
In this study, we hypothesize that the more temporally distant a reward is, the less likely the stimulus associated with it will be remembered. To test this, we devised an incidental memory-encoding task in which participants encode scene images preceded by a reward cue indicating when the reward will be delivered should the given trial be correct. We assume that the effect of reward modulation by TD will not be seen pre-consolidation but after consolidation, since dopaminergic facilitation of episodic memory is observed after 24 hours [6]. To validate this assumption, we divide the retrieval phase into two phases–pre-consolidation (referred to as ‘recent retrieval’ or 15 minutes after encoding) and post-consolidation (referred to as ‘remote retrieval’ or one week after encoding)–and acquire functional magnetic resonance imaging (fMRI) images from both retrieval phases to directly test whether the effect of temporally-discounted reward depends on dopamine-facilitated consolidation.
Materials and methods
Participants and materials
N = 26 volunteers participated in the study (12 women). Two subjects were excluded from the study due to failure to participate on the second day of the experiment. Two additional participants were excluded since they had at least one condition without a usable regressor for General linear modeling (GLM) analyses of fMRI data. The volunteers’ age ranged from 19 to 31, and their mean age was 25. Before study begin, all participants were screened if they matched the inclusion criteria of having no history of psychological or neurological disorder, not being pregnant, having no claustrophobia or tinnitus, and being right-handed. Only volunteers who met all the criteria were enrolled. All participants gave their informed written consent according to a document approved by the Institutional Review Board of Yonsei University. We acquired 22 complete fMRI datasets (244 trials of encoding, 404 trials of retrieval), and two participants’ data was partially lost due to malfunction of MRI scanner (1st participant: loss of 41 trials during first retrieval phase; 2nd participant: loss of 114 trials during first retrieval phase, and this participant’s data was not used for any analyses). The sample size of this study was determined by referring to the sample size of top-cited studies published during 2017–2018, which is 23–24 subjects [22]. Also, we note that our final sample size of 22 subjects falls within the range of recently published functional neuroimaging studies on episodic memory encoding and/or retrieval [23–25].
The experiment and data collection were performed via Cogent 2000 (Wellcome Trust Centre for Neuroimaging) based on MATLAB (The MathWorks; Natick, MA). For scene stimuli, 405 scene stimuli were sampled from a preexisting image database [26] or from online royalty-free images. All images were adjusted to a size of 92.1 mm x 92.1 mm and a resolution of 72 pixels. The order of visual stimuli presentation during the encoding and retrieval phase as well as the locations where the binary response options appear (left/right) were randomized. When the participants were placed inside the MRI scanner, visual stimuli were presented to them via MR-compatible goggles and their responses were recorded using an MR-compatible button-box.
Experimental paradigm
We devised a novel experiment which crosses reward with TD during memory encoding. The experiment consisted of three main sections, which were a scene-encoding task, a surprise memory-retrieval task, and an intertemporal decision-making task. Subjects performed these tasks while they were scanned for their brain activity inside an MRI scanner. The experiment took place for two days. The first day encompassed encoding and half of the retrieval task. The second day, one week later, comprised the remaining half of the retrieval and the intertemporal choice task (Fig 1). In the encoding task, participants classified whether a given scene was indoors or outdoors and were rewarded if they made the correct classification. The novel part of this classification task is that each trial is preceded by a cue indicating when the reward will be delivered should the following classification be correct, and there were four possible TDs: 0 days (same day), 1 day, 7 days, and 28 days. The reward magnitude was uniform across all trials, resulting in 5,000 Korean Won (KRW) when summed. For retrieval, scenes presented during the encoding task intermixed with new scenes (foils) were shown to the participants who decided whether they had seen the scene in the encoding task (old/new) and how confident they were about the decision (sure/guess). Finally, to estimate participants’ discount rate, the intertemporal choice task described in [17] was conducted, except that the reward was hypothetical in our study. This deviation can be ignored since no difference has been observed between real and hypothetical rewards in temporal discounting tasks [27].
Each trial in encoding and retrieval task was preceded with a jittered fixation. In the encoding task, correct response was followed by a green dot while incorrect response resulted in a red dot.
Experimental procedure
Upon arrival, participants were first asked to fill out the consent form and behavioral avoidance/inhibition scales for a measure of reward-sensitivity [28]. Then they were instructed about the scene-classification task, including the possibility of receiving an additional reward of up to 5,000 KRW depending on their classification accuracy and four possible time points when the reward will be delivered to them. Since the retrieval following encoding was intended to be a surprise memory test, participants were only informed that they will perform a simple cognitive task after the encoding phase. Then they were put inside the MRI scanner and a six-minute pre-encoding resting run was performed. After pre-encoding resting, participants performed six runs of encoding (scene-classification) task, with 44 items per each run, resulting in 264 scenes in total. Each condition (reward delivery after 0/1/7/28 day(s)) was comprised of 66 items. Then the post-encoding resting phase followed. After resting, participants were informed that the aforementioned ‘cognitive task’ is actually a recognition task and they performed the first half of the retrieval task inside the scanner, consisting of 202 items divided into five runs. After leaving the scanner, the participants were paid for the reward they earned during the encoding task.
Participants returned a week later and resumed the experiment by performing the intertemporal decision-making task three times. They also answered the Zimbardo Time Perspective Inventory (ZTPI) and an abbreviated 9-item form of Raven Standard Progressive Matrices (RSPM) [29,30] as a measure of their future orientation and intelligence, respectively. However, data collected from ZTPI and RSPM were not used for this study. Then the participants were positioned in the MRI scanner and first performed the identical intertemporal decision-making task which they performed outside the scanner. Afterward, the remaining half of the retrieval task took place, also consisting of 202 items distributed into five blocks. Finally, the structural images of participants (T1- and T2-weighted images) were acquired. The experiment terminated as the subjects were paid the rest of their participation reimbursement and provided their signature for receipt. Note that the bonus was bank-transferred according to each TD, so when returning to the experiment on the second day the participants had already received bonus for 0-days, 1-day and 7-days condition.
Behavioral data analyses
Participants’ task-specific performance during the encoding task was assessed by analyzing the response time (RT) and accuracy during scene classification. This analysis ensured that performance during the encoding task per se does not lead to differential memory performance according to conditions in the retrieval task. For retrieval tasks, participant’s memory performance and RT were assessed. First, participants’ memory performance for each condition was analyzed by calculating the corrected recognition (CR), which is a model-free memory measure calculating hit minus false alarm rate (‘old’ responses to targets minus ‘old’ responses to foils). Based on the previous finding that the reward-motivated subsequent memory effect (SME) is reflected in high-confidence responses [10], we additionally calculated high-confidence hits corrected for high-confidence false alarm rate. The CRs of participants were submitted to 2x4 repeated-measures analyses of variance (ANOVA). Second, participants’ RT was assessed for retrieval tasks to ensure that the results of the retrieval tasks are not explainable via RT.
Each participant’s discount rate was estimated based on the results of the delay-discounting decision-making task. A logistic function was fit to choices within each delay (D) to determine the indifference point, which refers to the amount of money which yields 50% probability of choosing either option. The indifference point for each delay was used to calculate the discounted value (DV):
Then, the DVs as a function of delays were fitted by a hyperbolic function [31], where
The estimated free parameter k was used as a measure for each individual’s impulsivity, where higher k leads to steeper hyperbolic curve or higher impulsivity. The purpose of deriving each participant’s k was to estimate the subjective value of delayed rewards in the encoding session tailored to each participant. This was done by multiplying objective numerical value for each trial (5,000 KRW/264 trials = 18.9 KRW) by the respective DV of each delay of each participant (18.9 KRW * DV). The obtained subjective value pertaining to each delay was entered as a parametric modulator for GLM of functional images acquired during retrieval runs.
Neuroimaging data acquisition and analyses
MRI data acquisition.
The MRI data for this study was acquired with a Phillips 3T MRI scanner using a 32-channel head coil. All functional images were T2*-weighted echo-planar images with repetition time of 0.8 s, voxel size of 2.4 x 2.4 x 2.4 mm, multiband factor of 6, and 60 slices in an ascending order, aligned to the anterior commissure–posterior commissure axis. The first ten images of each run were discarded for magnetic stabilization. A full dataset comprised of 500 volumes each for the pre- and post-encoding resting phase, 2520 (6 runs * 420 volumes per run) volumes for the encoding phase, and 3900 (2 phases * 5 runs * 390 volumes per run) volumes for the retrieval phase. For structural images, a T1-weighted MPRAGE (1 mm isotropic voxels) and a T2-weighted image were acquired. The T2 structural images were not used for any analyses in this study.
Preprocessing.
The acquired functional images were first unwarped using the Topup toolbox of FSL software (FMRIB, Oxford, UK) to correct for possible distortions due to magnetic field inhomogeneity [32]. Then the differences in the slice acquisition time for the functional images were corrected. Afterwards, the functional images were realigned to the first image of each run to correct for movement, and a mean functional volume was created to use as a representative for coregistration of functional and structural (T1) images. The coregistered structural image was segmented into grey matter, white matter, and cerebrospinal fluid according to tissue probability maps. The deformation fields acquired from this process were used for normalization of functional images into standard Montreal Neurological Institute (MNI) space, and during normalization the functional images were resampled into 2 mm isotropic voxel size. After normalization, the functional images were spatially smoothed by a Gaussian kernel (8 mm, full-width at half-maximum). All of the steps after Topup correction were performed via the SPM12 toolbox (The Wellcome Department of Cognitive Neurology, University College London, London, UK).
General linear modeling of functional images.
We performed whole-brain analyses of functional images using GLM. For GLM of encoding runs, we applied a boxcar function modeled at the beginning of the reward cue presentation extending to the end of scene encoding, resulting in 3.5 s in total, for our main regressor of interest (Table 1). Regarding GLM for retrieval runs, an impulse function was modeled at the beginning of the scene stimulus presentation. For regressors of no interest for both encoding and retrieval runs, we included impulse functions modeled to participants’ response of no interest (encoding runs: scene classification response; retrieval runs: sure/guess response), missed trials, first-order temporal derivatives for each impulse function, session constants and six motion regressors for every GLM. The rationale for employing a boxcar function for only encoding runs is to assume the cue and encoding phase as a single process.
For the group-level analyses of encoding runs, beta coefficients for 16 kinds of regressors of interest, which are the product of 4 (TD: 0, 1, 7, 28 days) x 2 (subsequently remembered/forgotten) x 2 (retrieved 15 minutes/7 days later) conditions, were forwarded to a group analyses where we performed analysis of variance (ANOVA). We tested eight comparisons: (1) main effect of subsequent memory (remembered vs. forgotten) within items retrieved recently, (2) main effect of subsequent memory within items retrieved remotely, (3) main effect of value (0 days > 1 day > 7 days > 28 days), (4) main effect of TD (28 days > 7 days > 1 day > 0 days), (5) 4 x 2 interaction of the subjective value of delayed reward (0, 1, 7, 28 days) and subsequent memory (remembered vs. forgotten) within recent retrieval, (6) 4 x 2 interaction of the subjective value of delayed reward (0 > 1 > 7 > 28 days) and subsequent memory (remembered vs. forgotten) within remote retrieval (7) 4 x 2 interaction of TD of reward (28 > 7 > 1 > 0 days) and subsequent memory (remembered vs. forgotten) within recent retrieval, and (8) 4 x 2 interaction of TD of reward (28 > 7 > 1 > 0 days) and subsequent memory (remembered vs. forgotten) within remote retrieval.
For GLM analysis of fMRI data for retrieval runs, we employed four kinds of regressors of interest: old stimuli which were correctly classified as old (i.e., HIT), old stimuli which were incorrectly classified as new (i.e., MISS), correct rejection of new stimuli, and junk regressors which were comprised of new stimuli incorrectly classified as old (false alarms) and no responses. For each ‘old’ regressors (HIT and MISS conditions), two parametric modulators were added according to the delayed reward it was associated during the encoding run: subjective value and objective TD. For subjective value, aforementioned parametric modulators derived by multiplying each trial’s object reward value by each delay’s discounted ratio of each participant (18.9 KRW * DV of each delay) were entered. On the other hand, the parametric modulators for the objective delay are 10−5, 1, 7, and 28 for 0 days, 1 day, 7 days, and 28 days, respectively. Since more than one parametric modulator was entered for one onset, the ‘order effect’ of entering multiple parametric modulators specific to SPM arises (see [33] for more details). Therefore, we performed two GLMs for each parametric regressor of interest by switching the order of parametric regressors entered to acquire statistics for each modulator, respectively (i.e., 1st GLM: subjective value as the first modulator, objective distance as the second modulator; 2nd GLM: vice versa). Finally, the statistical parametric map for parametrically modulated HIT of each participant acquired from 1st level analysis entered 2nd level analyses which performed t-tests against 0. We assumed that the significant voxels would reflect parametric neural activation according to each condition (subjective value or objective delay) during memory retrieval.
In addition to GLM, we performed small-volume correction (SVC) based on a priori regions of interest (ROI) using Harvard-Oxford atlas (Center for Morphometric Analysis). For encoding runs, we selected regions likely to be involved in reward-facilitated memory consolidation such as the HC [34] or scene-related memory processing such as the parahippocampal cortex [35]. We checked whether the striatum and the orbitofrontal cortex (OFC), regions known to be involved in value processing of monetary reward [10], are involved in representing the main effect of subjective value. Furthermore, for retrieval runs, we selected previously defined regions of the ‘retrieval success network’ such as frontoparietal control regions and the caudate as ROIs for HIT vs. correct rejection contrast [36].
The multiple comparisons issue upon reporting significant voxels from GLM was dealt with by applying the 3dClustSim function in AFNI (National Institute of Mental Health, Bethesda), in which we entered the median smoothness values of each participant’s residual maps and the grey matter mask as input. As a result, the function yielded a set of cluster thresholds that correspond to the alpha value of p < 0.05 in a one-tailed test and 18-connected cluster connectivity (edges and faces touch; NN = 2), following the default setting of SPM [37]. Entering residual maps from 22 subjects resulted in 102 contiguous voxels upon p-threshold of 0.001 or 2896 contiguous voxels under p < 0.05. The results of the following analyses follow the latter criterion for reaching the alpha value of p < 0.05.
Psychophysiological interaction.
The previous findings that the interaction between the HC and the neocortex during memory formation predicts successful encoding [38] and a recent study which found bidirectional informational flow between the neocortex and the HC during memory encoding and retrieval [39] led us to test the possibility of temporal-cue processing regions relaying their information to the HC during successful encoding. To do so, we performed a psychophysiological interaction (PPI) analysis on remotely retrieved trials, with the HC as a seed region. To acquire the variables needed for PPI, we performed an additional GLM on the encoding runs and the regressors of interest were ‘trials remembered upon remote retrieval modulated by TD (R-pmod)’ and ‘trials forgotten upon remote retrieval modulated by TD (F-pmod)’. We used R-pmod labeled as 1 and F-pmod labeled as -1 for the psychological variable, and eigenvariate time series of HC pertaining to R-pmod and F-pmod as a physiological variable. The interaction term was obtained by first deconvolving the physiological variable with the hemodynamic response function (HRF) to acquire the assumed neural response, then multiplying the neural response by psychological variables (R-pmod as 1 and F-pmod as -1), and convolving it with the HRF. These three variables (psychological variable, physiological variable and the interaction term) of interest and variables of no interest such as motion regressors and block effects were put into the GLM. The beta values pertaining to the interaction term in the GLM were entered into a group analysis.
Results
Behavioral results
First, we ensured that the task-related performance during the encoding task was independent of possible differences in memory performance among conditions by performing one-way ANOVA on RT and classification accuracy. First, RT among conditions did not show significant differences (mean ± standard deviation (SD); 0 days: 424.87 ± 30.22 ms, 1 day: 427.35 ± 33.67 ms, 7 days: 427.57 ± 31.95 ms, 28 days: 429.9 ± 27.61 ms). Second, the mean classification accuracy for each condition showed a ceiling effect (0 days: 93.46 ± 5.75%, 1 day: 92.63 ± 5.84%, 7 days: 92.01 ± 5.27%, 28 day: 93.25 ± 6.12%) as well as no significant differences among conditions. Therefore, in further analyses, we claim that the task-related performance of each condition during the encoding session does not account for possible differences in memory performance of each condition.
Participants’ CR was assessed by subtracting hit rate (the number of hits divided by total possible hits in the given condition) by false-alarm rate (the number of false alarms divided by total possible false alarms, identical across all conditions), regardless of the confidence rating for each decision (Fig 2). First, participants’ CR was subjected to a 2 x 4 ANOVA (recent vs. remote retrieval x 4 different delays of reward). Only the main effect of retrieval time point turned out to be significant (F(1,21) = 122.4, p < 0.001). Post hoc analysis of retrieval revealed that CR was significantly higher for items that were retrieved 15 minutes later in contrast to items retrieved one week later (t(22) = 8.64, p < 0.001), as expected. Although the main-effect of reward-delay conditions turned out to be insignificant in ANOVA using both sure and guess decisions as well as ANOVA using only sure decisions, we performed an exploratory t-test using only sure decisions. Here, we performed a paired t-test after concatenating 0 days and 1 day’s sure HITs as ‘sooner’ and 7 days and 28 days as ‘later’ conditions in order to make a straightforward interpretation. This discretization has been performed on previous temporal discounting studies [40], which is justified by the fact that participants’ choice behavior, which can be converted to subjective value, is linearly related to temporal delay. This exploratory t-test revealed that the ‘later’ condition’s CR is marginally higher than the ‘sooner’ condition CR (t(22) = 2.06, p = 0.05).
Error bars indicate standard deviation, 15 min = retrieval right after encoding, 1 week = retrieval 1 week after encoding, (left) CR using both sure and guessed responses (right) CR using only sure responses.
Participants’ discount rate (k) ranged from 0.0002 to 0.1112 (mean ± standard deviation (SD); 0.0209 ± 0.0309). When assuming subjective value for 0-day as 18.9 KRW, the subjective value for other days were all significantly lower than 0-day (1 day = 18.53 ± 0.53 KRW, t(21) = 3.28, p = 0.004; 7-days = 16.92 ± 2.41 KRW, t(21) = 3.86, p = 0.001; 28-days = 14.06 ± 4.38 KRW, t(21) = 5.19, p < 0.001).
Neuroimaging data
Replication of previous literature.
We performed a set of analyses to check whether results from previous studies can be replicated and to strengthen the results from the main analyses. In encoding runs, SVC yielded activation for the parahippocampal cortex pertaining to subsequently remembered vs. forgotten items retrieved one week later (MNI coordinates x, y, z = -34, -6, -22; pSVC = 0.003; k = 25; z = 2.75), and activation of the striatum (-24, -10, 8; pSVC = 0.009; k = 20; z = 2.37) and the OFC (32, 30, -22; pSVC = 0.012; k = 42; z = 2.27) when checking for regions showing a main effect of subjective value. For the first retrieval session, the caudate (8, 8, -4; pSVC < 0.001; k = 27; z = 3.79) was activated for retrieval success upon SVC. The same contrast applied to the second retrieval session which yielded whole-brain level activation in a cluster comprising the right inferior parietal lobule, precuneus, posterior cingulate, and a cluster of regions in the frontal lobe (Table 2). All these regions have been reported to be activated upon direct HIT vs. correct rejection contrasts [36]. SVC in addition to whole-brain analysis returned activation in the bilateral caudate (left: -12, 8, 8; pSVC < 0.001; k = 165; z = 3.64; right: 12, 6, 10; pSVC = 0.001; k = 201; z = 3).
Main analyses.
Among eight contrasts within encoding runs, only three contrasts yielded significant activations at the whole-brain level: (1) main effect of TD, (2) interaction of TD of reward (28 > 7 > 1 > 0 days) and subsequent memory (remembered vs. forgotten) within remote retrieval, and (3) interaction of TD of reward and subsequent memory within recent retrieval (Table 3). First, a cluster of regions encompassing the precentral gyrus, postcentral gyrus, superior temporal gyrus, insula, middle temporal gyrus, superior temporal gyrus, inferior frontal gyrus, cingulate gyrus, and the medial frontal gyrus showed activation in correlation with the TD of the reward, regardless of other conditions (Fig 3). Second, a cluster including the precentral gyrus, cerebellum, middle temporal gyrus, cingulate gyrus, caudate, anterior cingulate, inferior temporal gyrus, inferior occipital gyrus, precuneus, paracentral lobule, and the fusiform gyrus was activated upon searching for regions showing an SME proportionate to TD for items retrieved recently (Fig 4). Third, a cluster encompassing the frontal lobe, insula, and the precentral gyrus, and a cluster comprised of the temporal gyrus, frontal lobe, and the inferior parietal lobule showed an SME proportionate to the more distant TD for items retrieved remotely (Fig 5).
Color bar indicates t-statistics.
IE: SME proportionate to TD when retrieved recently. Color bar indicates t-statistics.
IE: SME proportionate to TD when retrieved remotely. Color bar indicates t-statistics.
Next, for analyses of retrieval runs, we examined areas showing proportionate activation to formerly associated value or TD of the retrieved stimuli. Significant whole-brain level activation was only found for TD during remote retrieval: a range of areas encompassing the bilateral superior frontal gyrus, middle frontal gyrus (MFG), and the medial frontal gyrus (Fig 6 and Table 4).
Color bar indicates t-statistics.
Since it was unexpected that the PFC represents the TD of reward associated with the stimuli during both encoding and remote retrieval, we performed a conjunction analysis to further clarify which regions are activated for both encoding and retrieval. Upon the threshold of p < 0.001 with at least ten contiguous voxels for conjunction, the MFG, insula, superior frontal gyrus, precentral gyrus, superior temporal gyrus, cerebellum, inferior frontal gyrus, and the medial frontal gyrus were activated (Table 5 and Fig 7). We performed a post hoc analysis to examine the SME of the MFG, the peak showing the greatest activation among regions identified by the conjunction analysis. The SME was defined as beta values of ‘remembered’ subtracted by ‘forgotten’ in each condition (TD). Pairwise t-tests revealed that activation related to the SME for reward without delay (0 days) was significantly lower than activation at 1 day (t(22) = 2.234, p = 0.037), 7 days (t(22) = 3.306, p = 0.003), and 28 days (t(22) = 4.75, p < 0.001). Furthermore, there was a trend that activation at 1 day is smaller than at 7 days (t(22) = 1.77, p = 0.09) and 28 days (t(22) = 2.48, p = 0.082). Therefore, the observed interaction of the SME between conditions was mainly driven by the significantly lower SME of the 0-days condition than by the other conditions.
Color bar indicates t-statistics, bar plot indicates SME (beta values of Remembered subtracted by Forgotten) of medial frontal gyrus (-42, 32, 0). Error bar indicates standard deviation.
These observations lead to two intermediate conclusions: (1) the greater the distance of the reward associated with a stimulus, the higher the activation of the MFG both during encoding and remote retrieval, and (2) this role of MFG becomes significant once consolidation takes place, given that this pattern of MFG does not emerge when items are retrieved before consolidation.
Psychophysiological interaction.
The HC seed used for the eigenvariate time series extraction was a 6 mm sphere around peak coordinate -22, -20, -20 obtained from R-pmod vs. F-pmod contrast in the whole-brain GLM. According to PPI results, regions including MFG and bilateral IPL showed functional connectivity with the HC (Table 6 & Fig 8).
Color bar indicates t-statistics. (Left) The seed region, the HC (-22, -20, -20), is represented as a green sphere.
Discussion
Reward-motivated memory encoding and temporal discounting of a delayed reward are two well-studied fields in cognitive neuroscience. But to our knowledge, no previous study has directly linked these domains. This study is the first to examine whether rewards delivered at different TD lead to differential memory effects. We found that temporally distant rewards might lead to better memory performance, which is in contrast to previous studies. We showed that a) when considering only confident retrieval (sure decisions), behavioral measure of memory performance (CR) showed a trend that the more reward is delayed, the higher is the chance of memory consolidation, b) in the encoding session, while the subjective value of the reward did not yield activation proportional to an SME as expected, TD did, and c) some of these regions were again observed during the second retrieval session when HIT activation was parametrically modulated by TD associated with the stimuli.
Our result seems to be in conflict with the preexisting literature on reward-motivated memory encoding and temporal discounting. While previous studies showed that a) reward is devalued as its delivery is delayed [14] and b) reward magnitude is positively correlated with the SME [10], our result suggests that a delayed reward acts as if it is more valued (when we put the emphasis on b) or—although reward is temporally discounted—a reward associated with TD might somehow involve a different mechanism than dopaminergic reward-modulation (when we put the emphasis on a). Since we have collected data from a delay-discounting decision-making task and found that participants showed temporal discounting behavior as reported in previous studies, it seems more plausible to assume the latter hypothesis that reward in a temporal context cannot be solely interpreted in light of previous reward-motivated memory literature. Thus, we seek a novel account to explain this unprecedented SME driven by temporally distant rewards.
To do so, we start by discussing the main effect of TD of reward during encoding that perceiving temporally distant reward engages lateral prefrontal and parietal activation (4th contrast of Encoding GLM; see Table 1). In fact, distant reward cues leading to lateral PFC or parietal activation are not unfamiliar. Lateral prefrontal activation has been reported after choosing delayed reward vs. immediate reward in intertemporal decision-making tasks [41,42]. However, linking these studies with the current results requires caution because intertemporal decision-making tasks typically involve situations in which participants are presented with larger-delayed and smaller-immediate rewards, thereby provoking self-control to inhibit impulsivity when participants choose the former over the latter, especially when taking into account that frontoparietal regions are representative members of the control network [43]. If lateral prefrontal activity reported in these studies is solely the product of executive control rather than TD, it would not support our results since our study does not require choosing between delayed and immediate rewards.
However, a study which examined the prediction of future rewards outside the intertemporal decision-making paradigm also reported that in the long vs. short delay contrast, the ventrolateral PFC (vlPFC), insula, dorsolateral PFC (dlPFC), dorsal premotor cortex, and the inferior parietal cortex are activated [44]. In addition, a study that examined single-neuron activity of rats revealed neurons in the OFC/ventromedial PFC (vmPFC) which fire more strongly for delayed than for immediate rewards [45]. Frontoparietal regions have been reported to be activated for processing time and future perception in general and it has been previously reported that elaborating a future event activates the inferior frontal gyrus and the supramarginal gyrus [46]. Additionally, studies on macaques have found activations in the dlPFC and the inferior parietal cortex [47] and the posterior parietal cortex [48] during time perception tasks. These comprehensive studies weaken the possibility that frontoparietal activation associated with choosing a distant reward in intertemporal decision-making tasks solely reflects self-control. Instead, they may indicate that frontoparietal regions also process some portion of temporal information upon perceiving a delayed reward.
Interestingly, in our study, some of these regions such as the PFC or the inferior parietal lobule (IPL) go further from merely representing the TD of reward (4th contrast of Encoding GLM) to showing an SME commensurate with the TD during memory encoding (8th contrast of Encoding GLM). This is supported by the PPI result that PFC and parietal regions are likely to be transferring their temporal information to the HC during encoding, though causality is not defined in our study. This is consistent with a previous study reporting that the PFC and the IPL are also members of the retrieval success network [36] as the PFC and the IPL are activated upon successful retrievals in contrast to correct rejections in memory retrieval tasks. While it appears to be counterintuitive to find the answer to the ‘encoding’ results from the ‘retrieval’ phase, the principle of transfer-appropriate processing supports this hypothesis [49,50]. It assumes that memory will be enhanced if memory retrieval reactivates similar regions as those that were engaged during cognitive operations of encoding. Cortical reinstatement, which refers to the reactivation of the original neural trace of a stimulus during retrieval [51], might be an example, since the degree of cortical reinstatement has been found to correlate with memory recall performance [52,53]. It has also been found that perception-retrieval overlap of the perirhinal cortex and the parahippocampal cortex predicts success recall for object- and scene-stimuli, respectively [54]. Given this background, it is not surprising that brain regions that respond to TD during encoding, which are also key regions contributing to retrieval success, enhance memory for items associated with distant reward by the coincidental encoding-retrieval overlap.
However, even if we assume that the SME of delayed reward is proportionate to the TD due to encoding-retrieval overlap of retrieval success (or ‘temporal-distance’) regions, it is conditional because this phenomenon is only observed when memory is retrieved one week but not 15 minutes later. This is an important argument because we are trying to provide an alternative account to the dopaminergic account, but the differential effect of reward on memory observed pre- and post-consolidation is what would be expected by the dopamine account. To resolve this, we focus on the fact that the remote retrieval took place one week after encoding in our study. Consolidation continues even after 24 hours, reorganizing the initially constructed engram of memory at a system (network) level [55]. One of the accounts regarding system consolidation of memory proposes that while initially formed memory is rich in episodic details, supported by the HC, it eventually becomes abstract and semantic over time, thereby becoming less dependent on the HC [56]. Indeed, an fMRI study over three months has shown that while recently formed memory relies more on hippocampal representations, systems consolidation shifts the site of memory from the HC to neocortical areas such as the mPFC over time [57]. In light of this, our results can be explained by the following: memory performance for recent retrieval is not different among conditions because while recently formed episodic details depend on hippocampal activation, the four temporal conditions did not show differences in HC activation. However, as the HC contributes less to the network, regions which initially represented cognitive information become the site of memory upon encountering the stimulus. In our study, the regions initially coding cognitive information (i.e., TD) such as the PFC showed differential activation for temporal conditions during encoding, and as a function of TD. This initial difference stood out more as memory becomes more dependent on traces in the PFC. This may be one of the reasons that stimuli associated with a more distant reward, triggering stronger activations in the neocortex, became more likely to be remembered as retrieval becomes less dependent on the HC.
This study is the first to examine how the TD of rewards modulates episodic memory and the dynamics of memory retrieval with time. This study is important because it suggested that temporally varied rewards may not act like rewards which vary in magnitude in the brain. However, this study has limitations because the behavioral effects are statistically weak. Furthermore, the results from this study could also be analyzed by pattern analyses such as multivariate pattern analyses so that representations among temporal conditions for encoding and retrieval can be directly compared.
Supporting information
S1 Table. Minimal raw behavioral data for all subjects used in this study’s analyses.
https://figshare.com/s/c34211f53f14c7bae391. Yoo, Jungsun; Min, Seokyoung; Lee, Seung-Koo; Han, Sanghoon (2020): SupportingInformation1_minimal_data. figshare. Dataset. https://doi.org/10.6084/m9.figshare.13285928.v1.
https://doi.org/10.1371/journal.pone.0249290.s001
(XLSX)
References
- 1. Eichenbaum H., Yonelinas A. P., & Ranganath C. (2007). The medial temporal lobe and recognition memory. Annu. Rev. Neurosci., 30, 123–152. pmid:17417939
- 2. Kim H. (2011). Neural activity that predicts subsequent memory and forgetting: A meta-analysis of 74 fMRI studies. NeuroImage, 54(3), 2446–2461. pmid:20869446
- 3. Hamann Stephan B., Ely Timothy D., Grafton Scott T., & Kilts Clinton D. (1999). Amygdala activity related to enhanced memory for pleasant and aversive stimuli. Nature Neuroscience, 2(3). pmid:10195224
- 4. Schultz W. (1998). Predictive reward signal of dopamine neurons. Journal of neurophysiology, 80(1), 1–27. pmid:9658025
- 5. Lisman J., Grace A. A., & Duzel E. (2011). A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP. Trends in Neurosciences, 34(10), 536–547. pmid:21851992
- 6. Miendlarzewska E. A., Bavelier D., & Schwartz S. (2016). Influence of reward motivation on human declarative memory. Neuroscience & Biobehavioral Reviews, 61, 156–176. pmid:26657967
- 7. O’Carroll C. M., Martin S. J., Sandin J., Frenguelli B., & Morris R. G. M. (2006). Dopaminergic modulation of the persistence of one-trial hippocampus-dependent memory. Learning & Memory, 13(6), 760–769. pmid:17142305
- 8. Rossato J. I., Bevilaqua L. R. M., Izquierdo I., Medina J. H., & Cammarota M. (2009). Dopamine Controls Persistence of Long-Term Memory Storage. Science, 325(5943), 1017–1020. pmid:19696353
- 9. Wittmann B. C., Schott B. H., Guderian S., Frey J. U., Heinze H.-J., & Düzel E. (2005). Reward-Related fMRI Activation of Dopaminergic Midbrain Is Associated with Enhanced Hippocampus- Dependent Long-Term Memory Formation. Neuron, 45(3), 459–467. pmid:15694331
- 10. Adcock R. A., Thangavel A., Whitfield-Gabrieli S., Knutson B., & Gabrieli J. D. E. (2006). Reward-Motivated Learning: Mesolimbic Activation Precedes Memory Formation. Neuron, 50(3), 507–517. pmid:16675403
- 11. Shohamy D., & Adcock R. A. (2010). Dopamine and adaptive memory. Trends in Cognitive Sciences, 14(10), 464–472. pmid:20829095
- 12. Botvinick M. M., Huffstetler S., & McGuire J. T. (2009). Effort discounting in human nucleus accumbens. Cognitive, Affective, & Behavioral Neuroscience, 9(1), 16–27. https://doi.org/10.3758/CABN.9.1.16.
- 13. Richards J. B., Zhang L., Mitchell S. H., & de Wit H. (1999). Delay or probability discounting in a model of impulsive behavior: effect of alcohol. Journal of the Experimental Analysis of Behavior, 71(2), 121–143. pmid:10220927
- 14. Green L., & Myerson J. (2004). A Discounting Framework for Choice With Delayed and Probabilistic Rewards. Psychological Bulletin, 130(5), 769–792. pmid:15367080
- 15. Kirby K. N., & Marakovic N. N. (1995). Modelling myopic decisions: Evidence for hyperbolic delay-discounting within subjects and amounts. Organisational Behavior and Human Decision Processes, 64, 22–30.
- 16. Myerson J., & Green L. (1995). Discounting of delayed rewards: Models of individual choice. Journal of Experimental Analysis of Behavior, 64, 263–276. pmid:16812772
- 17. Kable J. W., & Glimcher P. W. (2007). The neural correlates of subjective value during intertemporal choice. Nature Neuroscience, 10(12), 1625–1633. pmid:17982449
- 18. Kobayashi S., & Schultz W. (2008). Influence of Reward Delays on Responses of Dopamine Neurons. Journal of Neuroscience, 28(31), 7837–7846. pmid:18667616
- 19. Schultz W. (2010). Dopamine signals for reward value and risk: basic and recent data. Behavioral and Brain Functions, 6(1), 24. pmid:20416052
- 20. Enomoto K., Matsumoto N., Nakai S., Satoh T., Sato T. K., Ueda Y., et al. (2011). Dopamine neurons learn to encode the long-term value of multiple future rewards. Proceedings of the National Academy of Sciences, 108(37), 15462–15467. pmid:21896766
- 21. Mason A., Farrell S., Howard-Jones P., & Ludwig C. J. H. (2017). The role of reward and reward uncertainty in episodic memory. Journal of Memory and Language, 96, 62–77. https://doi.org/10.1016/j.jml.2017.05.003.
- 22. Szucs D., & Ioannidis J. P. (2020). Sample size evolution in neuroimaging research: An evaluation of highly-cited studies (1990–2012) and of latest practices (2017–2018) in high-impact journals. NeuroImage, 221, 117164. pmid:32679253
- 23. Xiao X., Zhou Y., Liu J., Ye Z., Yao L., Zhang J., et al. (2020). Individual-specific and shared representations during episodic memory encoding and retrieval. NeuroImage, 116909. pmid:32387627
- 24. Friedl-Werner A., Brauns K., Gunga H. C., Kühn S., & Stahn A. C. (2020). Exercise-induced changes in brain activity during memory encoding and retrieval after long-term bed rest. NeuroImage, 223, 117359. pmid:32919056
- 25. Miotto E. C., Balardin J. B., Martin M. D. G. M., Polanczyk G. V., Savage C. R., Miguel E. C., et al. (2020). Effects of semantic categorization strategy training on episodic memory in children and adolescents. PloS one, 15(2), e0228866. pmid:32069310
- 26. Konkle T., Brady T. F., Alvarez G. A., & Oliva A. (2010). Scene Memory Is More Detailed Than You Think: The Role of Categories in Visual Long-Term Memory. Psychological Science, 21(11), 1551–1556. pmid:20921574
- 27. Johnson M. W., & Bickel W. K. (2002). Within‐subject comparison of real and hypothetical money rewards in delay discounting. Journal of the experimental analysis of behavior, 77(2), 129–146. pmid:11936247
- 28. Carver C. S., & White T. L. (1994). Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: the BIS/BAS scales. Journal of personality and social psychology, 67(2), 319.
- 29. Bilker W. B., Hansen J. A., Brensinger C. M., Richard J., Gur R. E., & Gur R. C. (2012). Development of Abbreviated Nine-Item Forms of the Raven’s Standard Progressive Matrices Test. Assessment, 19(3), 354–369. pmid:22605785
- 30. Raven J. C. (1941). STANDARDIZATION OF PROGRESSIVE MATRICES, 1938. British Journal of Medical Psychology, 19(1), 137–150. https://doi.org/10.1111/j.2044-8341.1941.tb00316.x. pmid:13980735
- 31. Rodriguez M. L., & Logue A. W. (1988). Adjusting delay to reinforcement: comparing choice in pigeons and humans. Journal of Experimental Psychology: Animal Behavior Processes, 14(1), 105. pmid:3351438
- 32. Andersson J. L. R., Skare S., & Ashburner J. (2003). How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging. NeuroImage, 20(2), 870–888. pmid:14568458
- 33. Mumford J. A., Poline J.-B., & Poldrack R. A. (2015). Orthogonalization of Regressors in fMRI Models. PLOS ONE, 10(4), e0126255. pmid:25919488
- 34. Lisman J. E., & Grace A. A. (2005). The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory. Neuron, 46(5), 703–713. pmid:15924857
- 35. Staresina B. P., Duncan K. D., & Davachi L. (2011). Perirhinal and Parahippocampal Cortices Differentially Contribute to Later Recollection of Object- and Scene-Related Event Details. Journal of Neuroscience, 31(24), 8739–8747. pmid:21677158
- 36. Kim H. (2013). Differential neural activity in the recognition of old versus new events: An Activation Likelihood Estimation Meta-Analysis. Human Brain Mapping, 34(4), 814–836. pmid:22110008
- 37. Pauli R., Bowring A., Reynolds R., Chen G., Nichols T. E., & Maumet C. (2016). Exploring fMRI Results Space: 31 Variants of an fMRI Analysis in AFNI, FSL, and SPM. Frontiers in Neuroinformatics, 10. https://doi.org/10.3389/fninf.2016.00024.
- 38. van Kesteren M. T., Fernández G., Norris D. G., & Hermans E. J. (2010). Persistent schema-dependent hippocampal-neocortical connectivity during memory encoding and postencoding rest in humans. Proceedings of the National Academy of Sciences, 107(16), 7550–7555. pmid:20363957
- 39. Griffiths B. J., Parish G., Roux F., Michelmann S., van der Plas M., Kolibius L. D., et al. (2019). Directional coupling of slow and fast hippocampal gamma with neocortical alpha/beta oscillations in human episodic memory. Proceedings of the National Academy of Sciences, 201914180. pmid:31597741
- 40. Ballard K., & Knutson B. (2009). Dissociable neural representations of future reward magnitude and delay during temporal discounting. Neuroimage, 45(1), 143–150. pmid:19071223
- 41. Diekhof E. K., & Gruber O. (2010). When Desire Collides with Reason: Functional Interactions between Anteroventral Prefrontal Cortex and Nucleus Accumbens Underlie the Human Ability to Resist Impulsive Desires. Journal of Neuroscience, 30(4), 1488–1493. pmid:20107076
- 42. McClure S. M. (2004). Separate Neural Systems Value Immediate and Delayed Monetary Rewards. Science, 306(5695), 503–507. pmid:15486304
- 43. Spreng R. N., Stevens W. D., Chamberlain J. P., Gilmore A. W., & Schacter D. L. (2010). Default network activity, coupled with the frontoparietal control network, supports goal-directed cognition. NeuroImage, 53(1), 303–317. pmid:20600998
- 44. Tanaka S. C., Doya K., Okada G., Ueda K., Okamoto Y., & Yamawaki S. (2004). Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nature Neuroscience, 7(8), 887–893. pmid:15235607
- 45. Roesch M. R., Taylor A. R., & Schoenbaum G. (2006). Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron, 51(4), 509–520. pmid:16908415
- 46. Weiler J. A., Suchan B., & Daum I. (2010). When the future becomes the past: Differences in brain activation patterns for episodic memory and episodic future thinking. Behavioural Brain Research, 212(2), 196–203. pmid:20399235
- 47. Onoe H., Komori M., Onoe K., Takechi H., Tsukada H., & Watanabe Y. (2001). Cortical Networks Recruited for Time Perception: A Monkey Positron Emission Tomography (PET) Study. NeuroImage, 13(1), 37–45. pmid:11133307
- 48. Leon M. I., & Shadlen M. N. (2003). Representation of Time by Neurons in the Posterior Parietal Cortex of the Macaque. Neuron, 38(2), 317–327. pmid:12718864
- 49. Morris C. D., Bransford J. D., & Franks J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16(5), 519–533. https://doi.org/10.1016/S0022-5371(77)80016-9.
- 50. Ritchey M., Wing E. A., LaBar K. S., & Cabeza R. (2013). Neural Similarity Between Encoding and Retrieval is Related to Memory Via Hippocampal Interactions. Cerebral Cortex, 23(12), 2818–2828. pmid:22967731
- 51. Danker J. F., & Anderson J. R. (2010). The ghosts of brain states past: Remembering reactivates the brain regions engaged during encoding. Psychological Bulletin, 136(1), 87–102. pmid:20063927
- 52. Kuhl B. A., & Chun M. M. (2014). Successful Remembering Elicits Event-Specific Activity Patterns in Lateral Parietal Cortex. Journal of Neuroscience, 34(23), 8051–8060. pmid:24899726
- 53. Kuhl B. A., Rissman J., Chun M. M., & Wagner A. D. (2011). Fidelity of neural reactivation reveals competition between memories. Proceedings of the National Academy of Sciences, 108(14), 5903–5908. pmid:21436044
- 54. Schultz H., Tibon R., LaRocque K. F., Gagnon S. A., Wagner A. D., & Staresina B. P. (2019). Content Tuning in the Medial Temporal Lobe Cortex: Voxels that Perceive, Retrieve. Eneuro, 6(5), ENEURO.0291-19.2019. pmid:31451605
- 55. Dudai Y. (1996). Consolidation: Fragility on the Road to the Engram. Neuron, 17(3), 367–370. pmid:8816699
- 56. Nadel L., Winocur G., Ryan L., & Moscovitch M. (2007). Systems consolidation and hippocampus: Two views. Debates in Neuroscience, 1(2–4), 55–66. https://doi.org/10.1007/s11559-007-9003-9.
- 57. Takashima A., Petersson K. M., Rutters F., Tendolkar I., Jensen O., Zwarts M. J., et al. (2006). Declarative memory consolidation in humans: A prospective functional magnetic resonance imaging study. Proceedings of the National Academy of Sciences, 103(3), 756–761. pmid:16407110