Consolidation and restoration of memory traces in working memory

Psychon Bull Rev (2017) 24:1651–1657 DOI 10.3758/s13423-017-1226-7 BRIEF REPORT Consolidation and restoration of memory traces in working memory Sébastien De Schrijver 1 & Pierre Barrouillet 1 Published online: 1 February 2017 # Psychonomic Society, Inc. 2017 Abstract Consolidation is the process through which ephemeral sensory traces are transformed into more stable short-term memory traces. It has been shown that consolidation plays a crucial role in working memory (WM) performance, by strengthening memory traces that then better resist interference and decay. In a recent study, Bayliss, Bogdanovs, and Jarrold (Journal of Memory and Language, 81, 34–50, 2015) argued that this process is separate from the processes known to restore WM traces after degradation, such as attentional refreshing and verbal rehearsal. In the present study, we investigated the relationship between the two types of processes in the context of WM span tasks. Participants were presented with series of letters for serial recall, each letter being followed by four digits for parity judgment. Consolidation opportunity was manipulated by varying the delay between each letter and the first digit to be processed, while opportunities for restoration were manipulated by varying the pace at which the parity task had to be performed (i.e., its cognitive load, or CL). Increasing the time available for either consolidation or restoration resulted in higher WM spans, with some substitutability between the two processes. Accordingly, when consolidation time was added to restoration time in the calculation of CL, the new resulting index, called extended CL, proved a very good predictor of recall performance, a finding also observed when verbal rehearsal was prevented by articulatory suppression. This substitutability between consolidation and restoration suggests that both processes may rely on the same mechanisms. * Pierre Barrouillet [email protected] 1 Faculté de Psychologie et des Sciences de l’Education, Université de Genève, 40 bd du Pont d’Arve, 1211 Genève 4, Switzerland Keywords Working memory . Consolidation Working memory (WM) is thought to hold a limited number of short-lived representations (Baddeley, 1986). Although the sources of this rapid forgetting remain a debated issue (Barrouillet & Camos, 2015; Oberauer & Lewandowsky, 2013), there is a large consensus on the fragility of WM traces. Two classes of mechanisms have been identified that could overcome this inherent fragility. On the one hand, verbal rehearsal (Baddeley, 1986; Camos, Lagner, & Barrouillet, 2009) and attentional refreshing (Barrouillet, Portrat, & Camos, 2011; Cowan, 1992) have been assumed to counteract the effects of interference or temporal decay by restoring degraded memory traces. However, WM loss could also be counteracted by strengthening memory traces not after, but before, their degradation. This mechanism is called consolidation. Consolidation was first hypothesized in the context of long-term memory, where its existence is still a matter of debate (Brown & Lewandowsky, 2010). In the context of shortterm memory, consolidation has been conceived as the final step of encoding.1 After a first sensory encoding that provides input for later systems, perceptual encoding results in representations that can remain active as long as they receive bottom-up sensory input (Potter, 1993). The final step in which we are interested here consists of encoding perceptual representations into short-term memory by transforming volatile perceptual representations into a more durable storage, a process called short-term consolidation by Jolicœur and Dell’Acqua (1998). Using a dual-task paradigm in which 1 It should be noted that Ricker (2015; Ricker & Cowan, 2014) has suggested distinguishing between encoding and consolidation by introducing a mask at stimulus offset to put an end to perceptual encoding. We used such a procedure in the present study. 1652 either one or three letters were followed at various stimulus onset asynchronies by either a high or a low tone for a choice reaction time task, Jolicœur and Dell’Acqua (see also Stevanovski & Jolicœur, 2007) established that consolidation is a time-consuming process requiring central resources. Interestingly, they discarded the hypothesis that this process would simply be another form of verbal rehearsal. The reliance on central resources and independence from verbal rehearsal suggest some commonalities with the process of attentional refreshing through which memory traces are maintained in WM (Camos et al., 2009). The relationships between consolidation and refreshing have been recently addressed by Bayliss, Bogdanovs, and Jarrold (2015), who used complex span tasks in which lists of letters were presented for further serial recall, each letter being followed by a concurrent task (solving arithmetic problems or reading digits). The time available for consolidation was manipulated by varying the delay interval between each letter and the onset of the concurrent task, which intervened immediately after the letter (immediate condition) or following a delay of either 2,400 ms, in Experiment 1, or 1,000 ms, in Experiment 2 (delayed condition). Importantly, the concurrent task was followed by a second variable delay, of 0 ms in the delayed condition and 2,400 ms in the immediate condition (1,000 ms in Exp. 2), in such a way that the time elapsed between two successive letters remained constant across conditions. The time available for refreshing when performing the intervening task was manipulated by varying its cognitive load (CL), as defined by the time-based resource-sharing model (Barrouillet & Camos, 2015)—which is the proportion of time during which the task occupies attention, preventing refreshing activities from taking place. Several studies have demonstrated that the higher this CL, the lower the recall performance (Barrouillet et al., 2011). Two main findings arose from Bayliss et al.’s experiments. First, the delayed condition resulted in better recall performance than the immediate condition, even when articulatory rehearsal was prevented. Second, there was no interaction with CL; increasing the CL of the concurrent task had the same detrimental effect in both the delayed and immediate conditions. The authors concluded from this lack of an interaction that consolidation and refreshing are separate processes, and they suggested that consolidation has yet to be incorporated into WM models and into the mathematical function governing the trade-off between processing and storage (cf. Barrouillet et al., 2011). Bayliss et al.’s (2015) study is key, because it provides strong evidence for the existence of a process of consolidation in WM. Moreover, it suggests that consolidated memory traces remain affected by variations in CL. However, is the lack of an interaction between consolidation and CL indicative of the independence of consolidation and refreshing processes? This remains in dispute. Manipulating consolidation and refreshing opportunities amounts to manipulating the time Psychon Bull Rev (2017) 24:1651–1657 available for strengthening memory traces at two different points during the retention interval, before and after distracting events. Assuming a unique process with an effect on WM strength commensurate with its duration, there is no reason to suppose that successive interventions of this process could not have additive effects, and that manipulating their duration at two different points (i.e., before and after intervening activities) would lead to an interaction. Thus, Bayliss and colleagues might not have provided compelling evidence that consolidation and refreshing are two separate processes. Moreover, an assessment of the relationships between consolidation and refreshing was made difficult in Bayliss et al. (2015) by a peculiarity of the paradigm they used. When there was no delay for consolidation (immediate condition), extra time was added after the processing task to keep the overall retention interval constant across conditions of consolidation. This manipulation makes it difficult to assess the net effect of consolidation, because less time being available for consolidation was associated with more time being available for restoration, and vice versa. Additionally, the extra time after the processing task reduced its CL by increasing the time available for refreshing, but more importantly, it reduced the difference between the high-CL and low-CL conditions.2 Thus, despite the fact that large effects of CL were apparent across both the immediate and delayed conditions in Bayliss et al.’s study (ηp2 = .51 and .47 in their Exps. 1 and 2, respectively), it is possible that the absence of an interaction between consolidation and opportunities of refreshing was due to a reduced contrast between the two conditions of CL in the immediate condition, masking a possible greater susceptibility to interference for poorly consolidated memory traces. The aim of the present study was to reassess the relationship between consolidation and refreshing while removing this confound. For this purpose, we used a complex span task in which lists of letters were studied for further serial recall, each letter being followed by a parity task performed on four digits displayed successively on screen. As Bayliss et al. (2015) did, we manipulated both the time available for consolidation, by varying the delay interval between the presentation of the letter and the first digit to be processed, and the opportunity for refreshing, by varying the pace at which 2 Cognitive load can be expressed as the ratio between the amount of work to be done and the time available to do it (Barrouillet et al., 2011). Thus, increasing the time during which a task occurs reduces its CL. In Bayliss et al.’s (2015) Experiment 2, for example, four digits were presented at a rate of either 600 or 1,000 ms per digit, for high and low CL, respectively. Adding in both conditions an extra delay of 1,000 ms after the last digit as the authors did reduced the difference in CLs between the two conditions by increasing to the same extent the time available for refreshing in both conditions. Note that this is true only when CL is calculated from the start of the processing activity. If CL is calculated by taking into account the entire interletter interval, including the time available for consolidation (what we call in the following discussion the extended CL, or CLext), Bayliss et al.’s manipulations can be considered as leaving the CL unchanged. Psychon Bull Rev (2017) 24:1651–1657 the digits were presented on screen. However, contrary to Bayliss et al.’s procedure, these two factors were orthogonally manipulated, and no extra time was added after the processing task through variations of the delay for consolidation. In order to finely analyze the effect of consolidation time, each participant was presented with three durations of the delay interval (short, medium, and long). Half of the participants were presented with delays of 0, 1,000, and 3,000 ms. To avoid a confound between a very short consolidation and its absence (i.e., 0 ms of delay), other participants benefited from consolidation times of 500, 2,000, and 5,000 ms, defining two consolidation groups. Concerning the parity task, four different levels of CL were used. Beyond the expected detrimental effect of CL (Barrouillet et al., 2011), and in line with Bayliss et al. (2015), we expected to see better recall performance as the time available for consolidation increased. How the interplay between consolidation and refreshing times would affect recall performance remained an open issue. Method Participants One hundred fifty-nine undergraduate psychology students at the University of Geneva received partial course credit for participating (mean age = 21.49 years , SD = 4.57; 126 females, 33 males) and were randomly divided between the eight experimental groups resulting from a 4 (CLs) × 2 (consolidation groups) factorial design, with 18 participants per group (15 participants were excluded from the analysis for various reasons; see the Results section). Materials and procedure Lists of three to eight consonants were presented in ascending length for further serial recall, each consonant being followed by four digits displayed successively on screen for parity judgments after a delay interval for consolidation. Six different lists of consonants were created per length, with two lists for each of the three consolidation times that each participant was assigned (either 0, 1,000, and 3,000 ms or 500, 2,000, and 5,000 ms). For each length, these six lists were presented in a random order, with the attribution of lists counterbalanced across the consolidation time conditions. Each trial began by a ready signal for 1,000 ms, followed by the first consonant of the list. Each consonant was presented for 500 ms and followed by a 100-ms mask made of two superposed Xs. Then, depending on the consolidation condition, a blank screen of either 0, 1,000, or 3,000 ms, for one consolidation group, or 500, 2,000, or 5,000 ms, for the other, preceded the appearance of the first of four digits presented successively for 700, 900, 1,233, and 1,900 ms in the high, 1653 medium-high, medium-low, and low CL conditions, respectively; each digit was followed by a blank screen of 100 ms. Participants judged the parity of each digit by pressing appropriate keys until the end of the blank screen. The blank screen of the fourth digit was followed by the next consonant or by the word rappel (Brecall^), which cued participants to enter on the keyboard the list of consonants in the correct order. A stop rule was used, with the experiment ending when participants could not correctly recall any list at a given length. Before the experimental trials, participants were trained on the parity task with a series of ten digits presented at the same pace as in the forthcoming task. Six trials were allowed for them to reach an 80%-correct criterion. Participants were then trained on the complex span task at lengths 2 and 3, with three attempts to reach the 80%-correct criterion in the parity task. To ensure sufficient involvement in the task, a criterion was also used for the parity task during the experimental session, but it was relaxed to 70% correct due to the difficulty of the task. Spans were calculated per consolidation time condition (i.e., three distinct scores). Lists recalled in correct order were counted as half a point. Since we started at list length 3, the half points were added to 2 for each consolidation-conditionspecific score. Results Among the 159 participants enrolled in this study, 15 were excluded from the analyses either because they failed to reach the predefined criterion in the parity task during the training phase (8 participants) or the experimental session (two participants, at 67% and 66% correct), or because they did not recall correctly any list in at least two out of the three consolidation conditions, resulting in a floor effect (five participants). We performed an analysis of variance on the mean spans, with the two Groups of Consolidation Times and the four CLs as between-subjects factors, and Consolidation Condition as a within-subjects factor. This analysis revealed an effect of CL, with the mean span increasing as the CL decreased (4.92, 5.22, 5.62, and 6.25 for the high, medium-high, mediumlow, and low load conditions, respectively), F(3, 136) = 13.25, p < .001, ηp2 = .23. In line with Bayliss et al. (2015), we also uncovered an effect of consolidation time, revealed by two main effects. First, there was an effect of consolidation duration, F(2, 272) = 67.34, p < .001, ηp2 = .33. Second, the group benefiting from the longer delays of consolidation (500/ 2,000/5,000 ms) had better recall performance, F(1, 136) = 5.37, p < .05, ηp2 = .04, with no interaction between the two factors, F < 1. Accordingly, the mean spans increased with the time available for consolidation (4.80, 5.15, 5.45, 5.80, 5.70, and 6.10 for the consolidation times of 0, 500, 1,000, 2,000, 3,000, and 5,000 ms, respectively). Apart from small deviations probably due to sampling error (i.e., a slightly higher 1654 mean for the 2,000-ms than for the 3,000-ms condition), memory performance seems to be a monotonically decelerating function of consolidation time, with the same benefit of consolidation during the first second and across the four subsequent seconds (i.e., 0.65 in both cases, from 4.80 to 5.45 and from 5.45 to 6.10, respectively). Concerning the interaction between consolidation and CL, visual inspection of Fig. 1 suggests that the effect of consolidation time progressively decreased as the CL reduced. Thus, whereas increasing the consolidation time from 0 to 5,000 ms resulted in an increase in span of 2.14 in the highest CL condition (from 3.72 to 5.86), this increase was only of 0.44 in the lowest CL condition (from 6.08 to 6.53; Fig. 1). Nonetheless, whereas the interaction was significant in the group involving an immediate condition (0 ms of consolidation time), F(6, 120) = 2.30, p < .05, ηp2 = .10, it did not reach significance in the other group, F < 1. Overall, our results suggest that time for consolidation can to some extent compensate for a lack of opportunities of refreshing, and vice versa. For example, the highest CL condition preceded by a consolidation time of 3 s resulted in approximately the same span as a far lower CL condition (pace of 1,333 ms), but in the immediate condition (5.08 vs. 5.00). Thus, we investigated how consolidation time could be integrated within the mathematical function governing the tradeoff between processing and storage, as Bayliss et al. (2015) suggested. Recall that WM spans are a linear function of CL expressed as the proportion of time during which the concurrent task occupies attention. We tested the hypothesis of the complete substitutability of consolidation and refreshing by simply adding the time available for consolidation to the time available for refreshing, resulting in what we called extended cognitive load (CLext). Following the method used by Barrouillet, Bernardin, and Camos (2004) to approximate CL, we simply estimated a proxy of CLext as the number of digits to be processed per second for the 24 experimental conditions of our study. For example, in the highest CL condition, with a consolidation time of 2,000 ms, each digit was presented for 800 ms, resulting in a CLext of 4/(2 + 0.8*4) = 0.77. The mean spans observed on the 24 experimental conditions were regressed onto the corresponding CLext values, revealing a linear trend accounting for 92% of the variance observed in recall performance (Fig. 2). This proportion is lower but in the range of what had previously been observed for the relation between WM spans and CL (e.g., 93% in Barrouillet et al., 2004; 98% in Barrouillet et al., 2011). Extended CL proved a far better predictor than CL alone, estimated as the number of digits to be processed per second when consolidation time was not included in the calculation, or than consolidation time (50% and 32% of the variance accounted for, respectively). Although our results suggest some substitutability of the consolidation and refreshing processes, a potential caveat must be considered. The fact that we did not include a blank Psychon Bull Rev (2017) 24:1651–1657 delay interval following the processing activity, as Bayliss et al. (2015) did, reduced the confound between the different conditions of consolidation time and CL, as we explained above, but it introduced another confound, associated with the opportunity for verbal rehearsal after presentation of the letters and before entering the parity task. Apart from introducing the possibility of an effect of consolidation uniquely due to verbal rehearsal, this confound could make difficult the interpretation of the relationship between consolidation and refreshing, by making the delay interval between the letter and the parity task conducive to a mix between consolidation and rehearsal processes.3 To fix this problem, a control experiment was run in which we introduced the requirement for concurrent articulation. Control experiment For the sake of simplicity, this control experiment used the same task we previously presented, but with only two contrasted consolidation times (500 and 3,000 ms) and two values of CL (digits presented for 1,000 or 2,000 ms, for a high- and a low-CL condition, respectively) in a factorial within-subjects design. Twenty-four undergraduate students at the University of Geneva (mean age = 20.71 years, SD = 2.09; 20 females, four males) had to memorize lists of two to seven consonants presented in ascending length, with two lists per length in each of the four experimental conditions. Articulatory suppression was introduced by asking participants to repeat Bbadibu^ every 750 ms over each entire trial, from the appearance of the fixation point to the word Brecall.^ The stop rule and the scheme for calculating spans were the same as we previously described. The results revealed better recall performance with a long than with a short consolidation time (mean spans of 3.78 and 3.27, respectively), F(1, 23) = 10.51, p < .01, ηp2 = .31, and with low than with high CL (3.72 and 3.33, respectively), F(1, 23) = 5.61, p < .05, ηp2 = .26, but no interaction, F < 1 (Fig. 3). Beyond confirming Bayliss et al.’s (2015) observation of an effect of consolidation time even under articulatory suppression, these findings suggest that the Consolidation × CL interaction previously observed could have been related to the availability of verbal rehearsal. Once more, some substitutability of consolidation and refreshing appeared, since the long consolidation delay followed by the high-CL task and the short consolidation delay followed by the low-CL task resulted in spans that did not differ from each other significantly (3.65 and 3.52, respectively; t < 1). Although the restricted number of conditions makes this analysis less conclusive than the previous one with 24 experimental conditions, the CLext index was a better predictor of recall performance 3 We thank an anonymous reviewer for this suggestion. 1655 Mean span Psychon Bull Rev (2017) 24:1651–1657 Very fast 800 Fast 1000 Slow 1333 Very slow 2000 Encoding + ConsolidaƟon Ɵme (ms) Fig. 1 Mean spans as a function of cognitive load and the total time available for encoding and consolidating memory traces (i.e., 500 ms of presentation + 100 ms of mask + consolidation time), with logarithmic trends for each cognitive load condition (89% of variance explained) than the CL of the parity task (35%) or the consolidation time alone (62%). Discussion Mean span In this study, we aimed at investigating the relationships between the processes of consolidation and restoration in WM. First of all, our results confirm and extend those of Bayliss et al. (2015). As they observed, a delay between the presentation of each memory item and the forthcoming processing task had a positive effect on recall performance, with longer delays resulting in higher memory spans. Additionally, our extensive manipulation of consolidation delays specified this effect by revealing a decelerating function with an asymptote that probably reflects the maximum number of items that people can maintain for a given CL when memory traces have been encoded strongly. Like Bayliss et al. (2015), we interpreted these phenomena as providing evidence for a process of consolidation of WM traces taking place before the occurrence of the first distractor. Beyond a trend toward an interaction between consolidation time and CL, suggesting that more- R² = 0.92 Digits / sec. Fig. 2 Mean span in each of the 24 experimental conditions as a function of the number of digits per second, with time ratio serving as a proxy for CLext (extended cognitive load; see the text), along with the linear regression line and the associated R2 value Psychon Bull Rev (2017) 24:1651–1657 Mean span 1656 Low CL High CL ConsolidaƟon Ɵme (ms) Fig. 3 Mean spans as a function of the time available for consolidating memory traces and of cognitive load in the control experiment consolidated memory traces are less affected by decay and interference, it appeared that a lack of time for restoration (e.g., in high-CL conditions) can be compensated for by more time available for consolidation. This substitutability is reflected by the high predictiveness of the CLext index for span scores, which suggests that the time available for consolidating memory traces before processing distractors has approximately the same effect as free time available after this processing.4 Interestingly, as Bayliss et al. (2015) observed, this substitutability remained when verbal rehearsal was no longer available, showing that delays for consolidation have a beneficial effect on memory even under articulatory suppression. This fact points toward the existence of a consolidation mechanism relying on attentional resources that is independent of verbal rehearsal. Camos et al. (2009; Barrouillet & Camos, 2015) have proposed that verbal WM relies on two distinct and independent mechanisms for maintenance: verbal rehearsal and attentional refreshing. From the relation of substitutability we observed in the present study, a parsimonious hypothesis could be that some processes are shared between consolidation and restoration. Our results suggest that both rely on some attention-based mechanism for strengthening memory traces, whereas a possible role of verbal rehearsal in consolidation 4 Note that this substitutability is not perfect. In Bayliss et al.’s (2015) study, in which extra time was added after the processing task in immediate conditions in such a way that the overall retention period was kept constant, thus resulting in constant CLext, better recall performance resulted from the same amount of time being available before as after the processing task. Nonetheless, CLext still proved a better predictor than either consolidation time or CL alone (e.g., in their Exp. 2, 73%, 26%, and 31% of variance explained, respectively). remains to be established. Nonetheless, as Bayliss et al. claimed, current models of WM need to be modified to account for the effect of temporal factors associated with the consolidation of memory items before the occurrence of any distracting event. We suggest that the introduction of what we call the CLext index could be a first step in this direction. Author note This research was supported by a grant from the Swiss National Science Foundation N° 100014_156513 to P.B. References Baddeley, A. D. (1986). Working memory. Oxford, UK: Oxford University Press, Clarendon Press. Barrouillet, P., Bernardin, S., & Camos, V. (2004). Time constraints and resource-sharing in adults’ working memory spans. Journal of Experimental Psychology: General, 133, 83–100. doi:10.1037/0096-3445.133.1.83 Barrouillet, P., & Camos, V. (2015). Working memory: Loss and reconstruction. Hove, UK: Psychology Press. Barrouillet, P., Portrat, S., & Camos, V. (2011). On the law relating processing and storage in working memory. Psychological Review, 118, 175–192. doi:10.1037/a0022324 Bayliss, D. M., Bogdanovs, J., & Jarrold, C. (2015). Consolidating working memory: Distinguishing the effects of consolidation, rehearsal and attentional refreshing in a working memory span task. Journal of Memory and Language, 81, 34–50. Brown, G. D. A., & Lewandowsky, S. (2010). Forgetting in memory models: Arguments against trace decay and consolidation failure. In S. Della Sala (Ed.), Forgetting (pp. 49–75). Hove, UK: Psychology Press. Camos, V., Lagner, P., & Barrouillet, P. (2009). Two maintenance mechanisms of verbal information in working memory. Journal of Memory and Language, 61, 457–469. Psychon Bull Rev (2017) 24:1651–1657 Cowan, N. (1992). Verbal memory span and the timing of spoken recall. Journal of Memory and Language, 31, 668–684. Jolicœur, P., & Dell’Acqua, R. (1998). The demonstration of shortterm consolidation. Cognitive Psychology, 36, 138–202. doi:10.1006/cogp.1998.0684 Oberauer, K., & Lewandowsky, S. (2013). Evidence against decay in verbal working memory. Journal of Experimental Psychology: General, 142, 380–411. Potter, M. C. (1993). Very short-term conceptual memory. Memory & Cognition, 21, 156–161. 1657 Ricker, T. J. (2015). The role of short-term consolidation in memory persistence. AIMS Neuroscience, 2, 259–279. Ricker, T. J., & Cowan, N. (2014). Differences between presentation methods in working memory procedures: A matter of working memory consolidation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40, 417–428. doi:10.1037/a0034301 Stevanovski, B., & Jolicœur, P. (2007). Visual short-term memory: Central capacity limitations in short-term consolidation. Visual Cognition, 15, 532–563.