http://orcid.org/0000-0003-3373-7095
Figures
Abstract
Cueing facilitates retention and transfer of multimedia learning. From the perspective of cognitive load theory (CLT), cueing has a positive effect on learning outcomes because of the reduction in total cognitive load and avoidance of cognitive overload. However, this has not been systematically evaluated. Moreover, what remains ambiguous is the direct relationship between the cue-related cognitive load and learning outcomes. A meta-analysis and two subsequent meta-regression analyses were conducted to explore these issues. Subjective total cognitive load (SCL) and scores on a retention test and transfer test were selected as dependent variables. Through a systematic literature search, 32 eligible articles encompassing 3,597 participants were included in the SCL-related meta-analysis. Among them, 25 articles containing 2,910 participants were included in the retention-related meta-analysis and the following retention-related meta-regression, while there were 29 articles containing 3,204 participants included in the transfer-related meta-analysis and the transfer-related meta-regression. The meta-analysis revealed a statistically significant cueing effect on subjective ratings of cognitive load (d = −0.11, 95% CI = [−0.19, −0.02], p < 0.05), retention performance (d = 0.27, 95% CI = [0.08, 0.46], p < 0.01), and transfer performance (d = 0.34, 95% CI = [0.12, 0.56], p < 0.01). The subsequent meta-regression analyses showed that dSCL for cueing significantly predicted dretention for cueing (β = −0.70, 95% CI = [−1.02, −0.38], p < 0.001), as well as dtransfer for cueing (β = −0.60, 95% CI = [−0.92, −0.28], p < 0.001). Thus in line with CLT, adding cues in multimedia materials can indeed reduce SCL and promote learning outcomes, and the more SCL is reduced by cues, the better retention and transfer of multimedia learning.
Citation: Xie H, Wang F, Hao Y, Chen J, An J, Wang Y, et al. (2017) The more total cognitive load is reduced by cues, the better retention and transfer of multimedia learning: A meta-analysis and two meta-regression analyses. PLoS ONE 12(8): e0183884. https://doi.org/10.1371/journal.pone.0183884
Editor: Michael A. Motes, Center for BrainHealth, University of Texas at Dallas, UNITED STATES
Received: May 23, 2016; Accepted: August 14, 2017; Published: August 30, 2017
Copyright: © 2017 Xie et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data files of this study are available as Supporting information.
Funding: This research was funded by National Natural Science Foundation of China Grant (#31300864, #31771236) and Self-Determined Research Funds of CCNU from the Colleges’ Basic Research and Operation of MOE (#CCNU16A02023) to FW.
Competing interests: The authors have declared that no competing interests exist.
Introduction
As one of the material-oriented interventions in multimedia learning, cueing affects learners’ cognitive processing and learning outcomes [1–3]. The term cueing refers to the non-content information (e.g., arrows, color coding, highlighting) added in learning materials to attract learners’ attention and to further promote their selection, organization, and integration of instructional elements [4, 5]. A large body of research has shown that adding cues in multimedia materials facilitated retention and transfer of learning [4, 6–10]. Two recent meta-analyses have also confirmed the robustness of the cueing effect on learning outcomes [11, 12]. Many researchers have given an explanation from the perspective of cognitive load theory (CLT) [13], maintaining that cues could reduce learners’ overall cognitive load and help avoid overload, thus contributing to their learning performance. Nevertheless, two crucial problems that remain to be solved are (a) whether the cues really reduce total cognitive load, and (b) how a reduction in cueing-related cognitive load may be related to learning outcomes. In the current study, a meta-analysis and two following meta-regression analyses focused on these issues.
The measurement of cognitive load
Cognitive load is generally defined as a multidimensional construct representing the cognitive demands associated with performing a specific task [14, 15]. In the 1980s Sweller [16] proposed cognitive load theory, which articulated the association between cognitive resources and task demands in creating cognitive load. Learners will consume their cognitive resources as long as they are performing a task, leading to cognitive load. In this model, working memory is a cognitive resource, but is a limited one; only a small fraction of elements can be consciously handled per unit time, especially when they are novel or unfamiliar. However, long-term memory provides the ability to circumvent the limitation of working memory with the help of schemas. Schemas are cognitive constructs in which multiple elements are incorporated into a single element, which can then be processed automatically. Consequently, the construction and automation of schemas are the main goals of instruction.
In the field of educational research, cognitive load theory [13, 17] is mainly used to explain the effects of various forms of instructional design. According to this theory, intrinsic cognitive load (ICL) is not directly affected by instructional design. It is related to element interactivity in learning materials and learners’ prior knowledge. Element interactivity has been regarded as the primary, representative mechanism of ICL for quite a long time. The level of ICL of a specific task is usually assumed to depend on the level of element interactivity. An element can be anything that will be or has been presented, for example a concept or a procedure. Instructional materials with low element interactivity allow single (or several) element(s) to be processed with little or even no reference to other elements, thus resulting in a low ICL; however, high element interactivity materials contain elements that heavily interact with each other and cannot be processed separately, leading to a high ICL. Extraneous cognitive load (ECL) is concerned with the quality of instructional design. It is detrimental to the processes of schema construction and automation and thus will hinder learning. Germane cognitive load (GCL) is directly beneficial to learning. It can be imposed by the cognitive processes of active schema construction such as classifying, inferring, and organizing. The total cognitive load during information processing is the sum of the three kinds of cognitive loads. One important objective of instructional design is to ensure that the total cognitive load is within the learner’s cognitive capacity, in order to avoid cognitive overload [14].
Different techniques have been used to measure cognitive load, mainly including subjective rating scales, dual-task performance, and physiological measures [14, 18–20]. Pass [21] introduced the mental effort scale, which was a modified version of Bratfisch, Borg, and Dornic’s scale [22] for measuring perceived task difficulty. Pass’s 9-point mental effort scale included one item that asked learners to report how much mental effort they invested when learning the material. Since then, the mental effort or perceived difficulty scale has been widely used in research in the field of learning and instruction because it is easy to administer, is non-invasive, and has good reliability and validity [14].
Bratfisch, Borg and Domic’s [22] mental effort or perceived difficulty scale for measuring subjective cognitive load (SCL) has been modified in several ways in previous studies. This heterogeneity reflects differences in terms of scale items, number of scale units, and the timing of measurement [19]. Concerning items, Pass [21] asked participants to report their invested effort on a 9-point Likert scale by translating the perceived amount of mental effort into a numerical value. Other researchers have required learners to estimate how difficult (or easy) it was for them to learn from the instruction [8, 10, 23]. Some studies combined mental effort with perceived difficulty to measure cognitive load [24–27], whereas others added items measuring something that was different than cognitive load (e.g., “It was important to me to do well at this task” [28]). As for number of units, these scales have ranged from 9 points to 5 points (See Table 1). In most studies the questionnaire was given to the learners after learning had taken place [4, 8, 25], whereas other studies presented it during the learning process [29, 30].
What kind of cognitive load does mental effort or perceived difficulty (SCL) represent: ICL, ECL, GCL, or total cognitive load? We assert that mental effort or perceived difficulty is more likely to represent the total cognitive load for the following reasons: (1) Mental effort is defined as “the aspect of cognitive load that refers to the cognitive capacity that is actually allocated to accommodate the demands imposed by the task” [14]. It is influenced by three factors: task environment characteristics, learner characteristics, and their interaction. The combination of these factors is more likely to represent the total cognitive load, rather than a specific cognitive load. (2) A mental effort or perceived difficulty scale does not specify which aspect of the learning experience the participants are required to rate. Rather, participants report their overall feelings about learning. (3) This position is consistent with that of Pass and colleagues [14, 31, 32] and other researchers [33–35] who have considered all measures of mental effort or perceived difficulty to assess overall cognitive load, rather than its constituent components (i.e., intrinsic, extraneous, germane).
Effects of cueing on cognitive load and multimedia learning performance
Multimedia learning materials usually have high element interactivity, and learners often have no idea how to quickly search and process the correct elements in the limited time. In this context, the total cognitive load that learners bear can easily exceed the limited capacity of cognitive resources. In order to avoid cognitive overload during the learning process, many researchers add some non-content information (i.e., cues) in the learning material to guide the learners’ attention and reduce their total cognitive load. For example, Jeung, Chandler, and Sweller [7] studied participants who were learning geometry. In the experimental group, when the participants heard any comment about a certain rectangle (e.g., “area of the rectangle MADE”), this rectangle (e.g., MADE) in the picture would flash. De Koning et al. [4] added visual cues in the animation presented to the experimental group. When explaining a certain element in the cardiovascular system, all elements in the animation except this certain element would be darkened, which could be thought of as a spotlight-effect. Analogously, when experimental group participants in Lin and Atkinson’s [28] study were learning information about the rock cycle, red arrows would appear and point to the element being discussed. However, no such cues were presented to the control groups in these studies.
According to CLT, cues should prevent cognitive overload [4]. However, this has not yet received consistent support from empirical research. Some studies have found that cues could reduce students’ perceived amount of invested mental effort or perceived task difficulty. For instance, Kalyuga et al. (Experiment 2) [8] found that using color coding reduced searches for diagrammatic referents in the text and ameliorated split-attention effects, resulting in lower perceived difficulty. Berthold and Renkl [24] showed that participants who were provided color coding and flashing reported significantly less SCL than their counterparts without such aids. However, a larger number of studies have found that cues do not produce a significant reduction in SCL [29, 36, 37]. Ozcelik et al. [10] found there was no significant difference in the perceived difficulty of the instruction (chemical synapses) between the color-coded group and the conventional group. A series of studies by De Koning and colleagues also did not find any effect of cues on mental effort [4, 38–41]. In sum, there is a lack of cogent evidence about whether cues affect cognitive load.
Compared to research on cues and SCL, results concerning the impact of cues on learning outcomes can be considered to be a bit more consistent; that is, adding cues in multimedia materials can improve learning performance. Ozcelik, Karakus, Kursun and Cagiltay [10] discovered that color coding facilitated learners’ memory and comprehension of chemical synapses, revealing higher scores on retention and transfer tests, though the cues did not influence perceived difficulty. By now, a positive cueing effect on learning outcomes has been shown in numerous studies in various instructional domains, such as math [7], biology [4, 10, 39], engineering [42, 43], psychology [6], and physics [8, 44]. Though some studies did not find an improvement in learning outcomes due to cues [36, 38, 45], the authors of two recently published meta-analyses concluded that overall, cues indeed improved learning performance [11, 12].
Research questions and Hypotheses
The first question is whether cueing can indeed lower learners’ total cognitive load and help avoid cognitive overload, as cognitive load theory predicts. The present study used meta-analysis to assess whether cueing reduces cognitive load, as measured by SCL (mental effort, perceived difficulty, or their combination) and improves learning (retention and transfer). In addition, the present study calculated pooled effect sizes for the impact of cues on a retention test and transfer test in order to compare the results with those of two related meta-analyses [11, 12] that adopted different inclusion criteria from ours. Our first research question about effects of cueing on total cognitive load was not considered in the two recently published meta-analyses, which only looked at the effects of cueing on learning outcomes.
Supposing the findings of the meta-analysis do suggest that cueing reduces total cognitive load, then the following question is about the relationship between cueing-related cognitive load and learning outcomes. One of the solutions is to use correlation or regression analysis to explore this question. For example, a few studies have found a significant negative correlation between SCL and learning performance [23, 46]. Nevertheless, the vast majority of empirical researchers have separately analyzed the effects of cueing on cognitive load or on learning outcomes [4, 25, 29, 39], leading to a lack of evidence needed to make direct inferences about their relations. An alternative solution is to use meta-regression to examine the predictive effect of cueing-related cognitive load on learners’ performance. If total cognitive load is reduced by cues, there is every reason to predict better learning outcomes (e.g., better retention or transfer performance) according to CLT. Two meta-regression analyses were used to test this assumption. This statistical method synthesizes correlational results across studies. In this case the correlational data that were included were those describing the relation between cueing-related SCL and scores on retention or transfer tests, respectively.
According to CLT, we expected that (a) cues can reduce SCL (Hypothesis 1) and (b) promote scores on retention (Hypothesis 2a) and transfer (Hypothesis 2b) in multimedia environments, and further, (c) the more SCL is reduced by cues, the better retention (Hypothesis 3a) and transfer (Hypothesis 3b) of multimedia learning. Hypotheses 1, 2a and 2b were tested using meta-analysis; Hypotheses 3a and 3b were tested using meta-regression analyses.
Methods
The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were followed (S1 Checklist).
Literature search
To identify relevant studies on the effects of cueing in multimedia learning, a systematic literature search was conducted by searching the electronic databases PsycINFO, Education Research Complete, Science Direct, PubMed, ProQuest, and China National Knowledge Infrastructure (CNKI). Search engines such as Google Scholar and the reference lists of the identified studies were also used. The search keywords were “cue,” “cueing,” “signaling,” and “color coding” with different combinations of “multimedia learning,” “cognitive load,” “mental effort,” “retention,” and “transfer.” The search was limited to the period between January 1995 and March 2016. To minimize the file drawer problem, we (1) tried to contact some researchers to provide the details of their unpublished studies (e.g., dissertations or conference papers) through email, and also (2) assessed publication bias statistically (see below).
Study selection
Journal articles as well as dissertations and conference presentations would be selected. The following inclusion criteria for the retrieved articles were adopted for the meta-analysis. The studies were included if (a) they were based on an experimental design; (b) multimedia learning materials were used, that is, the materials simultaneously contained words (e.g., on-screen text, narration) and pictures (e.g., diagrams, videos, animations); (c) both an experimental group with cues and a control group without cues were compared; (d) sufficient quantitative data (e.g., means, standard deviations and n; t test or F test values) were reported to calculate effect size; and (e) the effect size did not go beyond three SD of the mean of all effect sizes to alleviate the effect of outliers representing extreme values [47], which would make mean values unrepresentative of the literature as a whole. Studies were excluded if they did not meet the inclusion criteria mentioned above.
It was important to emphasize that different studies might adopt multiple dependent variables to measure (total) cognitive load. A basic assumption of meta-analysis is the independence of effects, and the inclusion of multiple dependent variables in each study would not conform to this assumption [48]. Thus to abide by this assumption and avoid the potential deviation due to dependencies between effect sizes introduced by multiple variates per study, the following criteria were also used:
- The study measured SCL invested in the learning phase rather than in the test phase, regardless of whether the scale was presented after learning had taken place or during the learning process.
- We chose data representing mental effort during the learning process as long as they were provided independently, regardless of whether or not other indexes of cognitive load were available.
- If no data representing mental effort were included but data on the perceived difficulty of the instruction were provided independently, this result was included.
- If the only available data represented the combination of mental effort and perceived difficulty, rather than independent data for each, the combined result was included.
- Except for the three situations above, studies using any other measuring methods were disregarded. For example, Lin and Atkinson’s [28] study, which used a subscale named Effort to assess mental effort as well as other constructs (e.g., It was important to me to do well at this task.), was excluded from our study.
Data extraction
Data with respect to SCL ratings for both experimental and control groups were extracted by two of the authors (HX, YH) and checked by another (JC). Discrepancies were resolved through discussion.
Statistical analyses
As for analyses of retention and transfer performance, the included studies reported both SCL and a learning outcome (i.e., SCL and retention test, or SCL and transfer test, or SCL and both). Because the data included in this study were continuous data with no consistent unit, we chose Cohen’s d as the standardized estimate of effect size for each article [49]. Specifically, Cohen’s d was calculated as the mean score difference in SCL ratings or learning outcomes between cueing (experimental) and no-cueing (control) groups. When the studies reported multiple experiments or multiple conditions, the data were merged to compute one pooled effect size in order to minimize the deviation of results caused by a large number of effect sizes and disproportionate weight if not pooled [50]. For instance, Jamet et al. [25] conducted a 2 (Salience: salient, non-salient) × 2 (Display: sequential, static) between-subjects design, which could have generated two effect sizes (one effect size per condition of Display), but we combined these two into a pooled study-level effect size through CMA (see below). Study-level effect sizes were then averaged to obtain an overall average effect size. The 95% confidence interval (95% CI) of each effect size was also calculated. For Cohen’s d, the direction of the effect size was negative if the SCL rating or learning performance of the cueing group was lower than that of the no-cueing group. An effect size of ±0.2 was considered to be small, ±0.5 moderate, and ±0.8 large [51]. A random-effects model was preliminarily selected to calculate the pooled effect sizes and their 95% CIs because articles included in the present study differed in a number of variables (e.g., groups of participants, research methods), potentially resulting in a heterogeneity of results among studies.
Data were analyzed using the Comprehensive Meta-Analysis (CMA) 2.0 software (https://www.meta-analysis.com/). We calculated the Q statistic with its p value to test whether the random-effects model used in this study was reasonable. A statistically significant Q value indicates that it would be better to calculate effect sizes based on the random-effects model [48, 52]. Regarding publication bias, Egger’s linear regression test [53] was used. Through this test, a regression equation can be created with the standard normal deviate of each study as the dependent variable and the estimate’s precision of each study as the independent variable. The intercept of the regression equation provides a measure of publication bias. The smaller its deviation from zero the less pronounced the bias.
Results
Included studies
Results of the initial literature search and study selection are shown in Fig 1. A total of 32 articles that met the inclusion criteria were finally included and analyzed. There were 27 articles obtained from journals, 4 from dissertations, and 1 from an academic conference. Twenty-eight articles were written in English, and 4 in Chinese. Accordingly, 32 study-level effect sizes with respect to SCL were computed, involving 3,597 participants (See Table 1 and S1 Dataset). Twenty-five study-level effect sizes regarding retention containing 2,910 participants and 29 study-level effect sizes regarding transfer encompassing 3,204 participants were also computed.
Note. n = number of articles, k = number of effect sizes, RT = retention test, TT = transfer test.
Effects of cueing on SCL and learning outcomes
Table 2 presents the results of cueing on SCL as well as learning outcomes, and forest plots of the meta-analysis with respect to each index are shown in Fig 2. Concerning SCL, the meta-analysis based on the random-effects model revealed that the overall pooled effect size was small but statistically significant (d = −0.11, 95% CI = [−0.19, −0.02], p < 0.05). Thus is in line with CLT, adding cues in multimedia materials reduced learners’ perceived total cognitive load.
Note. (A) Meta-analysis of SCL. (B) Meta-analysis of retention test. (C) Meta-analysis of transfer test. Note. RT = retention test, TT = transfer test.
Concerning learning outcomes, there were small-to-medium cueing effects for both retention (d = 0.27, 95% CI = [0.08, 0.46], p < 0.01) and transfer (d = 0.34, 95% CI = [0.12, 0.56], p < 0.01). These results are consistent with the previous two meta-analyses; adding cues in multimedia materials facilitated retention and transfer of learning.
Potential relationships between cueing-related SCL and multimedia learning outcomes
If cues reduce SCL, would this lead to a better retention or transfer of learning as CLT would expect? To clarify this question, we conducted two parallel meta-regression analyses to investigate the potential relationships between SCL and scores for retention or transfer. In both analyses SCL (dSCL) effect sizes were used as the predictor, and retention test (dretention) and transfer test (dtransfer) effect sizes as dependent variables (Table 3; S2 Dataset). The data sources were the same as those used for retention- and transfer-related meta-analysis.
First, a meta-regression with dSCL as the predictor and dretention as the dependent variable was conducted (Table 3; Fig 3A). The result revealed that the effect of cueing represented by dSCL could significantly predict the effect of cueing represented by dretention (β = −0.70, 95% CI = [−1.02, −0.38], p < 0.001). Thus in line with CLT, the more SCL was reduced by cues, the better retention of learning. Similarly, a meta-regression with dSCL as the predictor and dtransfer as the dependent variable was conducted (Table 3; Fig 3B). The result showed that dSCL significantly predicted dtransfer (β = −0.60, 95% CI = [−0.92, −0.28], p < 0.001). Thus also in line with CLT, the more SCL was reduced by cues, the better transfer of learning.
Note. (A) Regression of dSCL on dretention. (B) Regression of dSCL on dtransfer. Note. RT = retention test, TT = transfer test. The size of the circle is proportional to study weight.
Publication bias analyses
As shown in Table 2, the p values of the Q statistic were significant (QSCL = 45.20, p < 0.05; Qretention = 147.02, p < 0.001; Qtransfer = 250.10, p < 0.001), indicating significant heterogeneity in the results. Thus, using a random-effects model in this study was appropriate. Egger’s linear regression test was not significant, suggesting that publication bias was an unlikely influence on the results.
Discussion
A meta-analysis and two subsequent meta-regression analyses were conducted in the present study to test whether the addition of cues can reduce learners’ subjective cognitive load and promote learning outcomes in a multimedia environment, and further, to explore the relationship between cognitive load and learning performance in this learning context. Consistent with Hypothesis 1, learners in the cueing condition reported a lower perception of cognitive load than learners in the no-cueing condition (d = −0.11, p < 0.05), revealing that cues could reduce subjective cognitive load. Results from retention-related meta-analysis and transfer-related meta-analysis also showed that adding cues in multimedia materials facilitated retention and transfer of learning significantly, thus supporting Hypotheses 2a and 2b. Our results were the same as the previous results on the transfer test. Both Richter et al. [11] and Xie et al. [12] found a small-to-medium cueing effect (r = 0.17 in the meta-analysis by Richter et al.; g = 0.36 in the meta-analysis by Xie et al.). Regarding the retention test, Xie et al.’s [12] meta-analysis and the current study both found that cues improved retention, but Xie et al. [12] found a medium-to-large cueing effect (g = 0.53) whereas our meta-analysis revealed a small-to-medium cueing effect. The reason for this discrepancy may be that we used different criteria than Xie et al. [12] did for including retrieved articles. Specifically, the articles included in the present study reported both SCL and learning outcomes, whereas Xie et al. [12] left SCL aside. Therefore the inclusion scope in our meta-analysis was different because studies with the measurement of retention scores but no SCL were excluded from our retention-related analysis. In addition, in support of Hypotheses 3a and 3b, dSCL negatively predicted both dretention (β = −0.70, p < 0.001) and dtransfer (β = −0.60, p < 0.001), suggesting that a lower cueing-related cognitive load meant higher scores on retention and transfer tests. Overall, results from the present study provided full support for CLT.
CLT is a plausible theory to explain the cueing effect. According to CLT, a reduction of subjective total cognitive load and an avoidance of cognitive overload would be expected in conditions with cues by virtue of their favorable external design, compared to conditions with no cues [5]. This assumption proved to be correct according to the results of our meta-analysis on the effects of cueing on SCL, retention and transfer. Thus, reducing cognitive load may be the crucial step in effective retention and comprehension in multimedia learning. However, to make a more direct inference about the cueing effect on learning outcomes based on the perspective of CLT, a further analysis of the relationship between cognitive load and scores of learning is probably requisite [23, 46]. Following CLT, the perceived total cognitive load would be negatively related to learning outcomes, such as retention and transfer performance [13]. Two meta-regression analyses in this study found this was indeed the case, again supporting CLT. These results were analogous to those of other empirical studies. Through two experiments, Huk et al. [23] discovered that students’ perceived cognitive load negatively correlated with both remembering and understanding. In Paik and Schraw’s study [46], learners were required to make a judgment of difficulty (JOD) after learning about a flushing toilet tank. Similarly, negative correlations were found between JOD and learners’ recall, as well as transfer performance.
Though the present study confirmed a reduction of total cognitive load (defined as subjective cognitive load in the form of mental effort, perceived difficulty, or their combination) in the cueing condition, it is seemingly impossible to draw conclusions about a specific type of cognitive load, i.e., ICL, ECL or GCL. Cueing is the consequence of external instructional design, but it may be related to both ECL and GCL [34]. For example, ECL would be decreased by reducing the unnecessary visual search when cues are provided in a given material (ICL is constant). However, it also could be that cueing increases GCL by optimizing the schema construction and automation.
It should be noted that the sensitivity of subjective ratings is critical for the measurement of cognitive load. SCL, especially mental effort, perceived difficulty, or their combination, is likely to be the most frequently used index of cognitive load [21, 38, 64, 65], but the sensitivity of measures of SCL has been questioned. After all, there are still quite a few empirical studies that have not found a reduction of SCL by cues as expected [4, 10, 36–41, 63]. There remain two possible explanations. First, there may be differences in participants’ understanding of the SCL questionnaire. Taking the question “How much effort did you have to invest to learn about the materials?” for example, some learners in the cueing condition might report that “I invested less effort because cues really worked” (as CLT would expect); others in the same condition could interpret the question in the opposite way, as “In an effort to comprehend the materials, I learned carefully” (not supporting CLT). Therefore, for the same question, learners in the cueing condition may respond differently from different viewpoints, probably decreasing the sensitivity of SCL.
Second, the effect of cueing on SCL may be moderated by external variables (e.g., dynamism of materials) that were not examined in the current analyses. For instance, cues have been found not to play a positive role in learning outcomes when the materials were dynamic [66]. Other studies have shown that when presenting materials in a static format, learners in the cueing condition outperformed those in the no-cueing condition [67]. A meta-analysis of cueing effects by Xie et al. [12] found that adding cues was beneficial for retention and transfer of knowledge when the multimedia material was static, whereas there were no significant effects on learning outcomes if the material was presented in a dynamic way. Taking CLT into consideration, these results could have been obtained because the elements in the dynamic materials overshadowed the effect of cueing on cognitive load, resulting in no learning improvement.
The current study presents the results of a meta-analysis adopting SCL as an index of cognitive load and scores on retention and transfer as indexes of learning outcomes based on CTL and, different from previous meta-analyses [11, 12], includes two meta-regression analyses focused on the relationship between SCL and learning outcomes, not only providing a more direct reference for CTL but also giving guidance on instructional design. Even so, several limitations should be acknowledged. First, we chose only SCL (specifically mental effort, perceived difficulty, or their combination) as the index of total cognitive load. Perhaps other measures would show additional unexpected but vital results. Second, we just focused on multimedia learning environments that contained both words and pictures. The effect of cueing when reading plain text (with no pictures) is also worthy of attention [12]. Third, though we argue that both mental effort and perceived difficulty measure the total cognitive load, it is not necessarily the case. After all, there are still many studies in which mental effort and perceived difficulty were used to measure ECL or GCL, leading to doubt about the measurement of cognitive load [19]. Moreover, differences between measurements of mental effort and measurements of perceived difficulty were ignored. It should be acknowledged that a learner’s rating of the perceived difficulty is not completely equivalent to the mental effort invested, though the two concepts are correlated. Van Gog and Paas [68] pointed out that mental effort was subordinate to a process, and contained more complex components than the task per se, whereas the perceived difficulty was mainly subject to the task. A potential example was that learners might not be motivated to invest sufficient mental effort when they perceived a specific task to be extremely difficult [68], leading to reversed scores of mental effort and the perceived difficulty scales. Thus, results concerning SCL in the present study must be treated with some caution.
All in all, in line with cognitive load theory, our meta-analysis and subsequent meta-regression analyses indicate that (a) cues can reduce subjective cognitive load, (b) cues can facilitate retention and transfer performance, and (c) the more SCL is reduced by cues, the better retention and transfer of multimedia learning. These results have clear theoretical and applied value.
References
- 1.
Mayer RE. Multimedia Learning. 2nd ed. New York: Cambridge University Press; 2009.
- 2. McCrudden MT, Rapp DN. How visual displays affect cognitive processing. Educ Psychol Rev. Forthcoming 2015.
- 3. Renkl A, Scheiter K. Studying visual displays: How to instructionally support learning. Educ Psychol Rev. Forthcoming 2015.
- 4. De Koning BB, Tabbers HK, Rikers RMJP, Paas F. Attention cueing as a means to enhance learning from an animation. Appl Cogn Psychol. 2007;21(6):731–46.
- 5. De Koning BB, Tabbers HK, Rikers RMJP, Paas F. Towards a framework for attention cueing in instructional animations: Guidelines for research and design. Educ Psychol Rev. 2009;21(2):113–40.
- 6. Jamet E. An eye-tracking study of cueing effects in multimedia learning. Comput Human Behav. 2014;32:47–53.
- 7. Jeung HJ, Chandler P, Sweller J. The role of visual indicators in dual sensory mode instruction. Educ Psychol (Lond). 1997;17(3):329–45.
- 8. Kalyuga S, Chandler P, Sweller J. Managing split-attention and redundancy in multimedia instruction. Appl Cogn Psychol. 1999;13(4):351–71.
- 9. Mautone PD, Mayer RE. Cognitive aids for guiding graph comprehension. J Educ Psychol. 2007;99(3):640–52.
- 10. Ozcelik E, Karakus T, Kursun E, Cagiltay K. An eye-tracking study of how color coding affects multimedia learning. Comput Educ. 2009;53(2):445–53.
- 11. Richter J, Scheiter K, Eitel A. Signaling text-picture relations in multimedia learning: A comprehensive meta-analysis. Educational Research Review. 2016;17:19–36.
- 12. Xie H, Wang F, Zhou Z, Wu P. Cueing effect in multimedia learning: A meta-analysis. Acta Psychologica Sinica. 2016;48(5):540–55.
- 13. Sweller J, Van Merriënboer JJG, Paas F. Cognitive architecture and instructional design. Educ Psychol Rev. 1998;10(3):251–96.
- 14. Paas F, Tuovinen JE, Tabbers H, Van Gerven PWM. Cognitive load measurement as a means to advance cognitive load theory. Educ Psychol. 2003;38(1):63–71.
- 15. Paas F, Van Merriënboer JJG. Instructional control of cognitive load in the training of complex cognitive tasks. Educ Psychol Rev. 1994;6(4):351–71.
- 16. Sweller J. Cognitive load during problem solving: Effects on learning. Cogn Sci. 1988;12(2):257–85.
- 17.
Sweller J, Ayres P, Kalyuga S. Cognitive Load Theory. New York: Springer; 2011.
- 18. Antonenko P, Paas F, Grabner R, Van Gog T. Using electroencephalography to measure cognitive load. Educ Psychol Rev. 2010;22(4):425–38.
- 19. De Jong T. Cognitive load theory, educational research, and instructional design: Some food for thought. Instr Sci. 2010;38(2):105–34.
- 20. Whelan RR. Neuroimaging of cognitive load in instructional multimedia. Educational Research Review. 2007;2(1):1–12.
- 21. Paas F. Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. J Educ Psychol. 1992;84(4):429–34.
- 22.
Bratfisch O, Borg G, Dornic S. Perceived item-difficulty in three tests of intellectual performance capacity (Rep. No. 29). Stockholm, Sweden: Institute of Applied Psychology; 1972.
- 23. Huk T, Steinke M, Floto C. The educational value of visual cues and 3D-representational format in a computer animation under restricted and realistic conditions. Instr Sci. 2010;38(5):455–69.
- 24. Berthold K, Renkl A. Instructional aids to support a conceptual understanding of multiple representations. J Educ Psychol. 2009;101(1):70–87.
- 25. Jamet E, Gavota M, Quaireau C. Attention guiding in multimedia learning. Learn Instr. 2008;18(2):135–45.
- 26. Moreno R. Optimising learning from animations by minimising cognitive load: Cognitive and affective consequences of signalling and segmentation methods. Appl Cogn Psychol. 2007;21(6):765–81.
- 27. Moreno R, Reisslein M, Ozogul G. Using virtual peers to guide visual attention during learning. J Media Psychol. 2010;22(2):52–60.
- 28. Lin L, Atkinson RK. Using animations and visual cueing to support learning of scientific concepts and processes. Comput Educ. 2011;56(3):650–8.
- 29. Tabbers HK, Martens RL, Van Merriënboer JJG. Multimedia instructions and cognitive load theory: Effects of modality and cueing. Br J Educ Psychol. 2004;74(1):71–81. pmid:15096299.
- 30. Ouwehand K, Van Gog T, Paas F. Effects of gestures on older adults' learning from video-based models. Appl Cogn Psychol. 2015;29(1):115–28.
- 31. Leppink J, Paas F, Van der Vleuten CPM, Van Gog T, Van Merriënboer JJG. Development of an instrument for measuring different types of cognitive load. Behav Res Methods. 2013;45(4):1058–72. pmid:23572251
- 32. Yung HI, Paas F. Effects of cueing by a pedagogical agent in an instructional animation: A cognitive load approach. J Educ Techno Soc. 2015;18(3):153–60.
- 33. Ayres P. Using subjective measures to detect variations of intrinsic load within problems. Learn Instr. 2006;16:389–400.
- 34. Jamet E, Fernandez J. Enhancing interactive tutorial effectiveness through visual cueing. Educ Technol Res Dev. 2016;64(4):631–41.
- 35. Song M, Bruning R. Exploring effects of background context familiarity and signaling on comprehension, recall, and cognitive load. Educ Psychol (Lond). 2015:1–28.
- 36. Crooks SM, Cheon J, Inan F, Ari F, Flores R. Modality and cueing in multimedia learning: Examining cognitive and perceptual explanations for the modality effect. Comput Human Behav. 2012;28(3):1063–71.
- 37. Wang F, Duan Z, Zhou Z, Chen J. The spatial contiguity effect in multimedia learning: The role of cueing. Acta Psychologica Sinica. 2015;47(2):224–33.
- 38. De Koning BB, Tabbers HK, Rikers RMJP, Paas F. Attention guidance in learning from a complex animation: Seeing is understanding? Learn Instr. 2010;20(2):111–22.
- 39. De Koning BB, Tabbers HK, Rikers RMJP, Paas F. Learning by generating vs. receiving instructional explanations: Two approaches to enhance attention cueing in animations. Comput Educ. 2010;55(2):681–91.
- 40. De Koning BB, Tabbers HK, Rikers RMJP, Paas F. Attention cueing in an instructional animation: The role of presentation speed. Comput Human Behav. 2011;27(1):41–5.
- 41. De Koning BB, Tabbers HK, Rikers RMJP, Paas F. Improved effectiveness of cueing by self-explanations when learning from a complex animation. Appl Cogn Psychol. 2011;25(2):183–94.
- 42. Boucheix J-M, Guignard H. What animated illustrations conditions can improve technical document comprehension in young students? Format, cueing and control of the presentation. European Journal of Psychology of Education. 2005;20(4):369–88.
- 43. Boucheix J-M, Lowe RK, Putri DK, Groff J. Cueing animations: Dynamic signaling aids information extraction and comprehension. Learn Instr. 2013;25:71–84.
- 44. Johnson AM, Ozogul G, Reisslein M. Supporting multimedia learning with visual signalling and animated pedagogical agent: Moderating effects of prior knowledge. J Comput Assist Learn. 2015;31(2):71–115.
- 45.
Arslan-Ari I. Examining the effects of cueing and prior knowledge on learning, mental effort, and study time in a complex animation [dissertation]. Texas: Texas Tech University; 2013.
- 46. Paik ES, Schraw G. Learning with animation and illusions of understanding. J Educ Psychol. 2013;105(2):278–89.
- 47. Li H, Li J, Li N, Li B, Wang P, Zhou T. Cognitive intervention for persons with mild cognitive impairment: A meta-analysis. Ageing Res Rev. 2011;10(2):285–96. pmid:21130185
- 48.
Lipsey MW, Wilson DB. Practical meta-analysis. Thousand Oaks, CA: Sage Publications; 2001.
- 49.
Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. New Jersey: Lawrence Erlbaum Associates; 1988.
- 50.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. Chichester, UK: Wiley; 2009.
- 51. Cohen J. A power primer. Psychol Bull. 1992;112(1):155–9. pmid:19565683
- 52.
Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. 2011. http://handbook.cochrane.org/.
- 53. Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–34. pmid:9310563.
- 54. Johnson AM, Ozogul G, Moreno R, Reisslein M. Pedagogical agent signaling of multiple visual engineering representations: The case of the young female agent. Journal of Engineering Education. 2013;102(2):319–37.
- 55. Spanjers IAE, Van Gog T, Wouters P, Van Merriënboer JJG. Explaining the segmentation effect in learning from animations: The role of pausing and temporal cueing. Comput Educ. 2012;59(2):274–80.
- 56.
Zhao H. Expertise effect in picture-text integration of multimedia learning: Evidence from eye movement [dissertation]. Wuhan: Central China Normal University; 2013.
- 57.
Zou Q. The effects of internal and external cues on multimedia learning: Evidences from eye movements [dissertation]. Wuhan: Central China Normal University; 2013.
- 58.
Tabbers HK, Martens RL, Van Merriënboer JJG, editors. Multimedia instructions and cognitive load theory: Split-attention and modality effects. Proceedings of National Convention of the Association for Educational Communications and Technology; Long Beach, CA; 2000.
- 59. Van Gog T, Jarodzka H, Scheiter K, Gerjets P, Paas F. Attention guidance during example study via the model’s eye movements. Comput Human Behav. 2009;25(3):785–91.
- 60. Jarodzka H, Van Gog T, Dorr M, Scheiter K, Gerjets P. Learning to see: Guiding students' attention via a Model's eye movements fosters learning. Learn Instr. 2013;25:62–70.
- 61. Seufert T, Brünken R. Cognitive load and the format of instructional aids for coherence formation. Appl Cogn Psychol. 2006;20(3):321–31.
- 62.
Zhou J. Effect of visuospatial cue on animation learning: Evidence from eye movement [dissertation]. Wuhan: Central China Normal University; 2014.
- 63. Amadieu F, Mariné C, Laimay C. The attention-guiding effect and cognitive load in the comprehension of animations. Comput Human Behav. 2011;27(1):36–40.
- 64. Paas F, Renkl A, Sweller J. Cognitive load theory and instructional design: Recent developments. Educ Psychol. 2003;38(1):1–4.
- 65. Paas F, Van Merriënboer JJG. Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. J Educ Psychol. 1994;86(1):122–33.
- 66. Kriz S, Hegarty M. Top-down and bottom-up influences on learning from animations. Int J Hum Comput Stud. 2007;65(11):911–30.
- 67. Jian Y-C, Wu C-J, Su J-H. Learners' eye movements during construction of mechanical kinematic representations from static diagrams. Learn Instr. 2014;32:51–62.
- 68. Van Gog T, Paas F. Instructional efficiency: Revisiting the original construct in educational research. Educ Psychol. 2008;43(1):16–26.