Introduction

Religious systems use various emotionally charged symbols to induce individual normative behavior. For example, in Judaism, the sound of the shofar(a ram’s horn) is a spiritual alarm, indicating it is time for people to repent and ask God for forgiveness. In the Muslim tradition, the call to prayer, athan, is a regular reminder to pray and re-establish the norms of communal life that Allah stipulated through his prophet Muhammad. Importantly, previous studies have suggested that such subliminal religious reminders induce prosocial and normative behavior. However, the research documenting the effects of religious priming on prosocial behavior has recently faced substantial criticism [1–3]. To contribute additional empirical data to this ongoing debate, we investigate the effects of religious auditory cues on individual honesty and extend the research literature on religious priming by adapting the methodology employed by Lang and colleagues [4]. Using a more generalizable sample and a less biased cheating task, we replicate important conceptual results observed by Lang and colleagues [4] yet fail to find evidence for the anticipated moderating effect of self-reported religiosity. Our results add important nuance to previous results by investigating how the efficacy of culturally specific religious primes are impacted not only by self-reported religiosity but also by religious socialization, identity, and ritual participation.

Perceptual cues associated with religion have been observed to affect decision-making across many behavioral domains [5–13] and their unconscious effects on ethical behavior have been studied extensively [1,14,15]. One particular behavioral domain studied in relation to religious cues is cheating. As an example, Aveyard [16] conducted two experiments investigating the effects of religious cues on ethical behavior in Middle Eastern participants. In the first experiment, participants unscrambled Arabic sentences with embedded religious or non-religious themes prior to taking a math test. The math test was incentivized, but unsupervised- so crucially, participants were able to dishonestly report their performance to earn more money. Aveyard [16]observed that participants’ honesty was unaffected by their exposure to religious (or non-religious) anagrams. However, in a second experiment, listening to the Islamic call to prayer, athan, was observed to impact reporting in the unsupervised math test. Specifically, participants that listened to the Islamic call to prayer were more honest than the control participants who did not listen to the call to prayer. These results indicate that auditory religious primes can affect individual behavior. Further, Aveyard’s findings suggest that individuals must have deep and natural associations with the sacred cue in order for it to change behavior.

Recently, however, religious priming has been the subject of a heated debate concerning the replicability of individual effects and the broader validity of this technique [1–3]. In a meta-analysis of 93 published and unpublished religious priming studies, religious priming was observed to have a consistent, small effect on the behavior of religious individuals [1]. These findings were contested as van Elk et al. [2] raised concerns that questionable research practices (QRPs) in social psychology confound meta-analysis results. They observed that religious priming effects are significant when Bayesian meta-analytic methods are used, but are non-significant when a Precision-Effect Testing–Precision-Effect Estimate with Standard Error (PET-PEESE) is used [2]. As these conflicting findings suggest, meta-analyses are not a panacea [17]; they are dependent upon the quality of the original data included and are subject to researchers’ degrees of freedom involved with conducting analyses and interpreting the results [18]. PET-PEESE methodology, for instance, is sometimes unreliable when the true effect size is small and when the meta-analysis includes twenty or fewer studies, studies that have small sample sizes, or studies that exhibit a large degree of heterogeneity of effect sizes [17,19]. Taken together, while meta-analyses are useful and needed tools for research, they are often inconclusive and should be supplemented with replication efforts, including both direct and conceptual replications [2].

To determine the validity and generalizability of religious priming, improved experiment procedures and more diverse samples must be explored. Oftentimes, religious priming research has utilized dictator games to measure prosociality [6,7,20–24]. However, behavior in dictator games has been observed to lack global generalizability, as people from different cultures demonstrate unique norms for altruism when playing dictator games [25,26]. The method of religious priming can also influence results as recent work indicates that explicit religious primes (i.e., writing tasks) produce only small effects on prosociality, while implicit religious primes (i.e., anagrams) do not appear to influence responses to prosocial measures [22]. Furthermore, religious priming effects have primarily been studied using largely homogeneous, WEIRD (Western, Educated, Industrialized, Rich, and Democratic) samples, raising additional concerns about the generalizability of observed effects across other populations [1,27,28]. Indeed, a cross-cultural study of 15 small-scale societies found that religious priming inconsistently impacts people across cultural and religious contexts [29; see also 4,27]. Consequently, replication attempts should test novel, yet conceptually-relevant prosociality measures that seek to avoid the limitations of existing measures and should investigate if an observed effect replicates in the same sample (typically Western university students), but also across various cultural and religious settings, where an effect is claimed to have cross-cultural validity.

To that end, the current study attempts to replicate and extend the findings from Lang et al. [4], examining whether priming with instrumental religious music would decrease the rate of participants’ dishonest behavior. Notably, this replication effort is distinct from what is sometimes referred to as a direct replication [30]. In this work, we adapt aspects of the methodology employed by Lang et al. [4] to investigate the conceptual consistency of their results. We designed this replication to provide results that would either increase or decrease the confidence in the interpretation of Lang et al. [4]. Therefore, this study utilizes the Open Science Framework (OSF) method of replication that, “reduces emphasis on operational characteristics of the study and increases emphasis on the interpretation of possible outcomes” [31]. (For a comparison of replication methodologies across disciplines, see Goméz and colleagues [30].)Below, we provide a succinct overview of Lang et al. [4]. Thereafter, we outline the unique aspects of this research and clarify the strategic motivations for modifying the procedure of Lang et al. [4]. Finally, in the Discussion, we detail how our adapted methodology may have limited our ability to replicate the findings observed in Lang et al [4].

Lang and colleagues [4]conducted a cross-cultural study on the effects of auditory cues on normative behavior using treatment with three musical stimuli (religious, secular, or white noise) prior to completing an arithmetic task designed to measure dishonest reporting for monetary reward (the Matrix Task, adapted from Mazar and colleagues [8]). Whereas they did not observe a main effect of musical treatment on dishonest reporting, the results revealed a Treatment*Religiosity interaction, such that participants self-reporting higher religiosity claimed less money upon hearing a religious musical track. Drawing from their results and previous research [16], Lang and colleagues [4] hypothesized that, “…instrumental music can serve as a reminder of normative behavior, but only for participants who previously formed an association between religion and specific music.” Here, we re-examine this learned association hypothesis and test the impact of exposure to religious symbolism (music) on cheating behavior. As opposed to studying various forms of voluntary sharing and altruism that vary across cultures, we examine cheating as an anti-social behavior that is frequently and explicitly targeted by religious norms [32–34]. Critically, this study builds on previous efforts in four distinct ways.

First, to strengthen the link between musical stimuli and their effect on dishonest reporting, the musical tracks were played on-loop, throughout the entirety of the experimental task (rather than 2 minutes before the task as in Lang and colleagues [4]). Second, we use a decision-making task that allows us to benchmark participants’ self-reported earnings against factually correct answers over 200 total trials (the Dots Game, adapted from Gino and colleagues [35]). Compared to the previously used Matrix Task, the Dots Game provides a less biased measurement of cheating and allows the participant more opportunities to report dishonestly, strengthening the signal without increasing time requirements. In the Matrix Task employed by Lang et al. [4], the researchers were unable to determine participant accuracy in a specific trial. Rather, Lang et al. [4] measured an aggregate of self-reported ‘correctly solved matrices’ to make calculated assumptions about cheating behavior. In comparison, the Dots Game records accuracy in each trial and, therefore, enables the direct measurement of beneficial mistakes (cheating), which decreases the likelihood that genuine, erroneous participant judgments will impact our model. Third, we add a no-music condition to compare the impact of not listening to any sounds at all. Finally, we broaden the generalizability of this research by diversifying the sample, adding an East Asian (Japan) site to our sample in which the religious traditions represented are non-exclusive and focus more on practice (orthopraxy) than belief (orthodoxy) [36,37]. Given the population size of Japan (126.2 million: [38]) and the prevalence of orthopraxic religion in East Asia [39], adding a Japanese sample enables further examination of the generalizability of the hypothesized religious priming relationships. For a detailed summary of the differences between the design employed in this paper and the design used in Lang et al. [4], see S1 Table H in S1 File.

Data were collected from three culturally distinct samples that differ in intriguing ways in their general religiosity levels: the Czech Republic, Japan, and the USA. While the majority of people in Japan [40] and in the Czech Republic [41] indicate that religion is not important in their lives, the majority of people in the USA think religion is important in their lives [42]. Further, only a minority of people in the USA are not affiliated to any religion (16%), while the majority of people in the Czech Republic (76%) and in Japan (57%) report no affiliation [43,44]. Despite their high level of non-affiliation, however, over 40% of Japanese people indicated that they believe in god and a majority (62.7%) indicated they go to religious places at least once a year [45]. Japanese religious systems are also broadly non-exclusive and there is substantial overlap in the ritual and festival practices performed in both Shinto shrines and Buddhist temples [37,46–49]. Hence, differences between these sites should enable useful comparisons of the hypothesized effect, as they introduce variation in levels of ritual participation, religious belief, and the significance of religious affiliations. Critically, the inclusion of a Japanese sample enabled us to test for the effect of using musical performances drawn from a distinctive non-Western musical heritage, echoing Lang and colleagues’ [4] choice to include Mauritian music.

Importantly, to date, there is only one published study that has investigated religious priming effects using a Japanese sample [50]. Miyatake and Higuchi [50] attempted a direct replication of Shariff and Norenzayan [6], utilizing identical methodology, and found that visual religious priming did not affect individuals’ pro-sociality. As Miyatake and Higuchi [50]suggest, these results may have been due to their decision to use primes that relied on the Western, Christian references traditionally employed in religious priming techniques. Their suggestion that “if local religions and culture had been reflected in the religious primes, the results might have been different” [50] supported our decision to select culturally relevant auditory cues for each sample (see Materials).

To isolate the effects of religious music, we designed four conditions: religious, secular, white noise, and control (no music). Following Lang et al.’s [4] learned-association supposition and their results, we predicted there would be no main effect of condition on dishonest behavior, but we expected an interaction between religiosity and condition. Specifically, we expected that participants higher in self-reported religiosity would behave more honestly than their less-religious peers but only when religious participants were listening to religious music. We also examined two supplementary hypotheses assuming: a) the moderating effects of affiliation to a religious organization that is congruent with the religious stimuli(religious affiliation) and b) the moderating effects of ritual participation frequency on the relationship between religious music and dishonest behavior.

The motivation for adding these supplementary hypotheses was to provide further nuance to previous findings by exploring the specific mechanisms that may facilitate the tentative effects of religious cues on normative behavior. The fact that Lang et al. [4] did not observe any main effect of condition on dishonest behavior and, consequently, that we do not expect such an effect in the present study supports the broader thesis that priming materials inherently connected to a specific cultural context will not affect people’s behavior indiscriminately. That is, to the extent that priming effects rely on symbolic communication, they should be detected only in people who are able to access the symbol’s conventional meaning and its connection to behavioral norms. Of course, some symbols include anthropomorphic characteristics that may exert additional effects on normative conduct. For instance, symbols with eyes may induce normative behavior because the presence of eyes, in general, indicates that one is being watched [13,51,52]. However, for most culturally specific cues(i.e., instrumental religious music), symbolic meaning and its association to moral norms must be learned and reinforced. In Lang et al. [4], the authors approximate individual reinforcement of religious music and its associated normative conduct by utilizing a concept of religiosity that subsumes dimensions such as religious belief, practice, experience, and commitment [53]. While this broad measure of religiosity is a useful and easy to use approximation, it does not afford for the precise estimation of an individual’s commitment to and understanding of culturally specific religious symbols.

Indeed, being religious or spiritual does not guarantee that one will understand the meaning of a symbol and its connection to the normative structure of a religious system. In multi-religious contexts (USA and Japan in our sample) or in contexts where people declare to be religious/spiritual but unaffiliated (Japan and the Czech Republic in our sample), people may believe in various supernatural agents and take part in many religious activities that are not directly connected to the religious system from which we sampled our priming material. In other words, participants who do not self-affiliate with religious organizations that practice the specific tradition of our religious stimuli may not have learned the conventional meaning and associations of the specific cue used in the present study (despite self-reporting high religiosity). Compared to religiosity, affiliation may serve as a more fine-grained predictor of the learned association between a religious cue and normative behaviors. To examine this idea in greater detail, we measured affiliation and tested its interaction with our treatment, anticipating that religious affiliation would enhance the effects of religious music on ethical behavior.

Furthermore, in many Western religious systems (i.e., Protestantism), religiosity is often associated with personal belief [54–56]. However, in other more orthopraxic oriented religious systems, the dominant dimension of religiosity may be ritual behavior. Together, collapsing orthopraxic and orthodoxic perspectives under the measure of religiosity obfuscates clear distinctions between the effects of practice and belief on behavior [54,57]. Critically, Lang et al. [4] suggested that it is through communal rituals, rather than belief itself, that the conventional association between a sacred symbol and norms are established and perpetuated. Relatedly, previous findings have observed that increased frequency of ritual behavior facilitates favorable treatment of other co-religionists in real-life and in laboratory contexts, and across various economic games [4,12,58–60]. Music is central to many religious rituals [61,62], serving several functions like coordination and synchronization [10,63,64], but also creating lasting associations between ritual context and normative conduct. Importantly, Lang and colleagues [4] observed a relationship between frequency of ritual participation and the effectiveness of an auditory religious prime on moral behavior. Thus, to provide a more nuanced investigation of this issue, we also included a measure of ritual participation frequency and interacted this measure with our manipulation, expecting that frequent ritual participation would strengthen the effects of religious music on honest behavior.

Materials and methods

Results

All data were analyzed in R (version 3.4.3, R Core Team 2017). We first constructed an Ordinary Least-Square Regression (OLS) model with treatment as a factor variable, investigating the main effects of musical condition on the percentage of dishonestly claimed earnings. We set the religious condition as a reference category to compare its effects with various controls (secular music, white noise, control), while holding the effects of age, gender, and site constant as simple fixed effects. Note that the USA was set as a reference category for the site factor variable; however, this selection was arbitrary and did not affect any of the main estimates of interest. Next, three interaction OLS models were constructed, looking at the moderating effects of religiosity, ritual participation, and religious affiliation on the relationship between the treatment and dishonest behavior. Religious affiliation was determined using religious organization affiliation and religious identity responses. Specifically, a participant was considered affiliated if they belonged to a religious organization and self-identified with the religion associated with the religious stimulus at each site (i.e., Christianity in the Czech Republic and US, Shinto in Japan). Note that while Lang and colleagues [4]used beta regression to model the percentage data in their previous analyses [69], the current results from OLS regressions are qualitatively similar to the results of beta regression; hence we opted for simpler models. The results from beta regression models are reported in the, S1 Table D in S1 File. Likewise, since our participants were nested within sites, it would be more appropriate to use linear mixed models to investigate our main hypotheses. However, given that there are only three categories in our nesting variable, estimating individual site intercepts from the partially pooled data did not yield qualitatively different results compared to using sites as simple fixed effects (see S1 Table E in S1 File). Additionally, we adjusted our models to account for: the perceived difficulty of the dots task, the difference between the average trial completion time, and the completion times for trials where participants cheated. In the final robustness check, we hold constant the ratings of musical stimuli to ensure that the observed effects were not caused by differences between the stimuli’s perceived affect, tempo, or impact (see S1 Table F in S1 File).

Manipulation check

Perceived sacredness was observed to be significantly different between musical conditions [F(2,299) = 53.8]. Across all sites, the religious track received significantly higher ratings of sacredness than did the secular(β = -0.85; 95% CI = [-1.21, -0.49]) or white-noise tracks (β = -1.91; 95% CI = [-2.27, -1.54] see Table 1 for descriptive statistics). Similar results were obtained with the secular/religious measure [F(2,300) = 40.34], showing higher religiosity ratings of the religious song compared to the secular (β = -1.32; 95% CI = [-1.69, -0.95]) or white-noise tracks (β = -1.60; 95% CI = [-1.97, -1.23]).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Descriptive statistics of aggregate unethical behavior and post-experiment ratings of musical stimuli.

https://doi.org/10.1371/journal.pone.0237007.t001

Dishonest behavior

Dishonesty in the Dots Game was observed by measuring the proportion of inaccurate higher paying (right side) selections made. This dishonesty metric was calculated by dividing the number of times a participant inaccurately indicated that the higher paying side contained more dots by the number of trials (120) in which the lower paying (left side) truly contained more dots. Participants across all sites earned an average of $5.86 (SD = $1.52), indicating right side incorrectly on average in 27.39% of trials (SD = 29.21%). Interestingly, the average rates of dishonest reporting differed between our sites: while in the Czech Republic, participants claimed on average 11.69% (SD = 13.72%) incorrectly, in Japan and the USA the rates were as high as 29.88% (SD = 28.43%) and 40.56% (SD = 34.28%), respectively (see Fig 1B).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1.

Mean values for dishonestly claimed earnings with 95% CIs divided by condition (A) and site (B). Each level displays a bar with 95% CIs and a density plot.

https://doi.org/10.1371/journal.pone.0237007.g001

Looking at the distribution of dishonest reporting across our musical treatment, we observed that the control condition had the lowest amount of cheating, followed by the religious, white-noise, and secular conditions (see Table 1 and Fig 1A). However, these raw results ignore the hierarchical structure of our data where participants are nested within sites. Hence, to examine the effects of our treatment on dishonest reporting more rigorously, we regressed the incorrectly claimed right sides on our musical treatment, holding the site-specific mean levels of dishonest reporting constant. This regression model revealed that there were no substantial differences between the religious and the secular (β = 3.45; 95% CI = [-4.02, 10.91]), white-noise (β = 3.21; 95% CI = [-4.21, 10.62]), or control conditions (β = -4.91; 95% CI = [-12.37, 2.55]; see Table 2). The inability to find differences between conditions replicates Lang and colleagues’ [4] previous finding.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Estimates with 95% CIs from Ordinal Least Squares regressions for the percentage of higher-paying side (right) claimed as having more dots.

https://doi.org/10.1371/journal.pone.0237007.t002

Following the absence of treatment main-effect, we tested three moderator models, investigating the role of self-declared religiosity, ritual frequency, and religious affiliation. First, we did not observe an interaction between condition and self-reported religiosity. While an increase in self-declared religiosity predicted decrease in the proportion of incorrectly reported right sides in the religious music condition (β = -2.12), this decrease was imprecisely estimated and the 95% CI crossed zero (-6.77, 2.53). Furthermore, this religiosity coefficient was not substantially different from coefficients in the secular (β_difference = 2.97; 95% CI = [-3.40, 9.35]), white-noise (β_difference = -1.40; 95% CI = [-7.94, 5.14]), or control conditions (β_difference = 0.82; 95% CI = [-5.49, 7.13]; see Fig 2A). For site-specific results, see S1 Table A in S1 File.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Interaction plots with predicted values for dishonestly claimed earnings and 95% CIs.

A. The regression slope of religiosity in the religious condition did not differ from the other conditions. B. Ritual frequency predicted decreased cheating, and the regression slope differed from other condition (albeit the 95% CI crossed zero for the difference between the religious and secular conditions). C. Self-declared affiliation to religious organization congruent with our stimuli predicted decreased cheating in the religious condition but not in the other conditions. We display 95% CIs only for the religious condition for easier reading. All CIs are displayed in Table 2.

https://doi.org/10.1371/journal.pone.0237007.g002

Following the effects of ritual participation found in Lang et al.[4], we built a second moderator model measuring the effect of the ritual attendance frequency and its interaction with treatment. An increase of one on the ritual frequency scale (0 –Never, 6 –More than once per week) predicted a decrease of roughly 4 percentage points in dishonestly claimed compensation in the religious condition (β = -4.19; 95% CI = [-7.34, -1.03]). Importantly, the slope of the ritual attendance differed between conditions, showing that ritual attendance decreased the ratio of incorrectly claimed right sides only to a small extent in the secular condition(β_difference = 3.46, 95% CI = [-1.01,7.94]), and had no effect in the noise(β_difference = 4.49, 95% CI = [-0.10, 9.08]) and control conditions(β_difference = 3.89; 95% CI = [-0.48,8.26]). See also Table 2 and Fig 2B and S1 Table B in S1 File for site-specific results.

Finally, in the third moderator model, we analyzed the effects of self-reported affiliation matching the specific religious stimulus at each site(binary yes/no variable) and its interactions with the musical treatment. Religious affiliation had a strong negative relationship with dishonest reporting in the religious condition, predicting roughly 26 percentage points lower number of unfairly claimed compensations (β = -26.09; 95% CI = [-40.12, -12.06]). Importantly, we observed a significant Condition*Affiliation interaction: the effect of religious affiliation was much weaker in all three remaining conditions (Secular: β_difference = 23.78, 95% CI = [5.64, 42.90]; Noise: β_difference = 21.84, 95% CI = [2.76, 40.92]; Control: β_difference = 24.94, 95% CI = [5.97, 43.91]; see Table 2 and Fig 2C). These findings indicate that participants who affiliated with a religious tradition matching our stimuli were less dishonest when listening to the religious song, but affiliation had no effect in the remaining conditions (see S1 Table C in S1 File for site-specific results). See also, S1 Table D in S1 File for a robustness check of these results using the Beta regression and S1 Table E in S1 File for using linear mixed models (these robustness checks support our findings obtained with simpler models reported here).

As a final robustness check, we also adjusted our models for the mean completion time of the dots task, perceived difficulty of the task, and musical characteristics of our stimuli (see S1 Table F in S1 File). Across all models, we observed that the rates of dishonest behavior were predicted by faster completion times on the dishonestly reported trials. That is, the less time participants dedicated to decision making, the more likely they were to report dishonestly because correct answers required deliberately counting the dots and making sure one selected the correct answer. This finding is congruent with perceived difficulty of the task, which negatively predicts the number of incorrectly reported dots. Notably, these findings can be explained by dishonest participants’ willingness to pre-determine the higher compensation choice (hold down the ‘M’ key), which would naturally decrease participation time and task difficulty. After adjusting our models for these variables, the moderating effects of ritual participation frequency and religious affiliation remained stable. Furthermore, we assessed whether the reported results hold even after the models accounted for the musical characteristics of our stimuli: tempo, influence, positivity, and negativity (see S1 Table F in S1 File). While perceived negativity of the played track predicted dishonesty, we still observed the effect of ritual and religious affiliation as well as the interactions of Condition*Affiliation and Condition*Ritual, albeit we could not assess the effects of musical characteristics in the control condition due to the fact this condition had no musical stimulus. These findings indicate that the religious condition tracks did not affect cheating in the Dots Game due to their musical characteristic ratings; rather, participants who affiliated with a religious organization and participated in rituals were uniquely affected by the sacred music primes and, in turn, played less dishonestly than unaffiliated participants.

Discussion

In the present study, we conducted a cross-cultural replication and extension of religious priming research by Lang et al. [4]. Specifically, we tested the hypothesis that auditory religious cues decrease unethical behavior in cheating games compared to other auditory cues (secular, white noise, and control). We collected data on university populations from three countries with distinct religious and cultural norms: the Czech Republic, Japan, and the United States of America. All participants played a cheating game, the Dots Game, during which they listened to a musical track and had an opportunity to maximize their earnings by playing dishonestly. There was no main effect of musical treatment on cheating behavior. Further, we were unable to replicate the interaction effect of Condition*Religiosity observed by Lang et al. [4]. However, we did observe interactions between condition and religious affiliation and condition and ritual participation. Auditory religious cues were found to decrease dishonest reporting for participants frequently attending religious services and for those affiliated with a religious organization matching our religious stimuli.

Taken together, our results generally provide support for Lang and colleagues’ [4] observation that exposure to religious music is not enough to prime honest behavior in all contexts. Notably, our findings provide additional evidence for a mechanism they proposed; specifically, an entrenched association between music and the religious values it reinforces is required for the activation of normative behavior. In their paper, Lang & colleagues [4] observed that situational religious factors, such as reported religiosity and ritual participation, played a role in the activation of religious cues and facilitation of their effects on ethical behavior. Although we did not observe the Condition*Religiosity interaction in the present research, we did observe similar interactions indicating that individual religious characteristics play a role in how people react to religious auditory cues.

In order to understand why we were unable to replicate the Religiosity*Treatment interaction, it is important to consider two points. First, compared to the Mauritian sample in Lang et al. [4], our Japanese sample included greater religious diversity. All Mauritian participants were Hindu in Lang et al. [4], hence the selected religious music was generally familiar and associated with their affiliated tradition. In our Japanese sample, however, participants from a variety of religions were represented (Christianity, Buddhism, Shinto, Judaism, atheism, unknown, and other). While Gagaku music is sometimes performed as part of Buddhist religious ceremonies, it is most strongly associated with Shintoism. Therefore, it is possible that for some of our participants this music did not cue normative behavior typical for the targeted religious affiliation. Second, religion in Japan, as elsewhere in Asia, is typically regarded as non-exclusive and has been described as having a ‘practical’ orientation wherein adherence to religious beliefs is seen as of secondary importance to ritual practices [37,39,70], potentially confounding the measure of religiosity. For instance, Kavanagh and Jong [39], recently demonstrated that while only 10% of 1,000 Japanese respondents self-identified as religious, 34% in the same sample identified as Buddhist, 5% as Shinto, and 33% endorsed the existence of God.

The increased religious diversity of our sample implies that religiosity in the current paper has a distinct meaning as it is understood within the context of an individual’s religious background. Indeed, the site-specific analyses of the Condition*Religiosity interaction (see S1 Table A in S1 File) revealed that in Japan, religiosity had the smallest effect on dishonest behavior in the religious condition (albeit these coefficients were imprecisely estimated). This finding is in line with similar research showing that in multi-religious societies, cross-religious symbols may reduce trust and cooperation [71, but see also 72].

Secondly, the relationship between affiliation, ritual participation, and religiosity is complex. It is entirely possible for someone to be religious/spiritual and, at the same time, be unaffiliated with a specific religion or religious organization. In the USA, for instance, where religiosity is typically understood as affiliation to a specific church [73], more than a quarter (27%) of adult respondents indicated they were spiritual, but not religious [42]. However, if a person is spiritual without being affiliated to a specific religious institution, they may lack the necessary associations with the sacred auditory cues we selected and therefore could be less, or entirely, unaffected by our stimulus. To make the relationship clearer, we coded affiliation to specific religious organizations and demonstrated that membership in the matching tradition had an effect. We conjecture that this is due to people affiliated with a religious organization having more exposure to sacred music in meaningful contexts than unaffiliated peers and consequently, affiliated persons are likely to have stronger associations to the moral implications of religious auditory cues than the unaffiliated. In support of this, research on exposure and learning has demonstrated that a person’s affective response to a stimulus can change over time [74]. In addition to repeated exposure, the meaningfulness of a stimulus can affect a person’s response to it [75]. With repeated exposure to meaningful stimuli, people are able to create associations and have the ability to strengthen their judgments about the stimuli and its propositions [76].

It is also important to consider the fact that unaffiliated religious people often have differing beliefs about the connection between specific religions, belief in God(s), and morality [77]. The majority of religiously affiliated people in the USA (55%) agree that the belief in God is necessary to be moral; conversely, however, of all adults—affiliated and unaffiliated—the majority in the Czech Republic (78%), Japan (55%), and in the USA (56%), do not think it is necessary to believe in god to be moral [44,78]. Based on previous literature and on the findings of the present research, we suggest that religious auditory stimuli cue ethical behavior only for participants who believe that their affiliated religion is inherently linked to morality.

This interpretation is congruent with the results from the regression model of the interaction between condition and the frequency of ritual attendance, where ritual attendance has the largest negative effect on dishonest reporting in the religious condition. Through repetitive reminders of sacred cues during religious ceremonies, the association between a specific sacred cue and the moral doctrine of a specific religion is strengthened and the cue is emotionally charged with special significance [61]. This in turn may result in a larger influence over participants’ behavior being exerted upon perception of the cue [79,80]. Importantly, collective rituals are also a public venue for communicating commitment to supernatural agents and the norms they impose on believers [80–82]. However, the commitment is not signaled only to other believers but also to oneself as a form of auto-signaling reassuring participants about their beliefs in supernatural agents [58,83,84]. Hearing religious music may be a subconscious reminder of participation in rituals, strengthening the auto-communicated commitment signals. While the effect of ritual participation was weaker compared to religious affiliation (similar to the findings of Lang et al. [4]), we still detected a signal supporting this interpretation.

It is important to note that our analyses do not take into account what type of rituals our participants attend as well as the frequency with which our religious stimuli are played during specific religious ceremonies; hence, the signal is noisy. A larger, high-powered sample allowing for testing a three-way interaction between treatment, religiosity, and affiliation to specific religious organization may solve this issue in future studies. Furthermore, our sample consisted predominantly of university students. While the sample was culturally and religiously diverse, it generally lacked diversity in age or employment status. Moreover, due to specific lab guidelines, participants in the Czech sample received an additional reward of points that were redeemable for a course credit, which may have partially decreased their motivation to report dishonestly for monetary compensation. Indeed, this motivation may have acted as a boundary condition for religious priming across samples. As previously noted by other researchers using priming techniques [23,29], there may have been little room to observe religious priming effects if the motivation to be dishonest was relatively low in the Czech Republic. To avoid these problems, cross-cultural researchers should be careful to align laboratory incentive structures across samples. Moreover, future studies on priming with religious music should consider sampling from more populations with cultural and demographic diversity that would allow for the assessment of between-site differences in the hypothesized effects (see S1 Tables A-C in S1 File). Likewise, since stimulus selection can influence Type I error rates if stimuli are not representative of their theoretical construct and are treated as fixed-factors [85], the selection of locally salient religious stimuli should be made more robust by including at least three different stimuli at each site. Future researchers should also explicitly control for stimulus variation by employing mixed models that treat both participants and stimuli as random factors [86].

Additionally, it is useful to consider the limitations of single-item scales when interpreting our results [87]. The reliability of the religiosity measure we employed was not clearly established in this research. This measure contained only a single-item and was not pre-tested between sites prior to experimentation [87]. Therefore, it may have been the case that religiosity was understood and expressed idiosyncratically within each of our cross-cultural samples. Furthermore, the religiosity scale was asymmetrical as the word “spiritual” was not included at both ends of the scale (see Questionnaire materials in S2 File). This scale asymmetry was consistent across all sites and was due to an error in implementing the materials of Lang et al [4]. Together, the lack of pre-tested validity of our single-item measure as well as the scale’s asymmetry, may have limited our ability to replicate the primary Condition*Religiosity interaction observed in Lang et al [4]. To explicitly address such concerns, future researchers should investigate the reliability of different religiosity scales across cultures, peoples, and religions.

Finally, it is worthwhile to discuss a limitation of the Dots Game procedure. The Dots Game has been utilized to measure ethical decision-making preferences for nearly a decade [35] (See Materials for a review of the procedure). In this research, we use the Dots Game to observe cheating behavior as it provides a less biased- and more granular- measurement than the Matrix task employed by Lang and colleagues [4]). Furthermore, we investigated religious priming effects on honesty because normative regulations of altruism and sharing may vary across cultures substantially more compared to norms regulating cheating [27,29]. Despite the benefits of using the Dots Game (see in-depth discussion in Introduction), our results may be influenced by signal detection biases (for a review of signal detection research, see [88–90]). Due to the compensation scheme, participants may have preferred the higher-paying (right) side and therefore, may have had an unconscious bias to detect more dots on the higher-paying side.

Indeed, research has shown that Dots Game participants attend more to the higher-paying side in incentivized Dots Games [65]. Unconscious attentional bias may increase ambiguity, making it less clear to participants when their choices are inaccurate. Thus, cheating behavior may have been facilitated by an interaction of signal detection bias with various factors(e.g., noise, perceived difficulty or distraction). Greater familiarity with a musical track, for example, could have increased participants’ bias for detecting higher-paying side dots, which in turn may have increased participants’ likelihood to report inaccurate higher-paying selections. However, given the potential for third variable problems, we adjusted our models for perceived difficulty and musical ratings (See Supporting Information for an in-depth review). Furthermore, it is unlikely that unconscious signal detection biases can fully explain the cheating behavior we observed in this paper, as other findings indicate that Dots Game participants are at least partially conscious of their unethical behavior [65].

We recommend that future research extends the religious priming literature by exploring methods that will provide less biased cheating data. Specifically, future studies should test the religious priming effects on a broader spectrum of samples, including horticulturalists and pastoralists from small-scale societies [29]. Moreover, we encourage future researchers to independently investigate if the differences observed between our samples replicate. For instance, cross-cultural researchers could attempt to replicate and extend the understanding of the uniquely low cheating rates observed in our Czech sample, as well as the depressed Condition*Religiosity interaction coefficient observed in the Japan model. Additional, experiments could test religious priming effects on cheating behavior across a variety of cheating tasks. Finally, we encourage healthy science practices and invite researchers to improve the generalizability of our findings by iterating on our replication with pre-registered designs. In support of such future endeavors, we have made our study materials and data available for access on the Open Science Framework (https://osf.io/k4dt8).

Conclusion

In summary, we conceptually replicated findings in the religious priming literature that indicate sacred cues affect individual ethical behavior and support a learned association hypothesis [4]. Although we did not find evidence for the expected interaction effect between religiosity and musical primes, we did observe an interaction effect between ritual participation and musical prime and between religious affiliation and musical prime. More specifically, sacred auditory cues were found to affect ethical behavior for individuals who attend religious rituals or were affiliated with a religious organization that practices the tradition associated with the relevant musical cues. These indirect priming effects were congruent with Lang and colleagues [4], even though the current paper extended the research design by utilizing a decision-making task better able to detect cheating (the Dots Game) and by including a non-Western site with greater religious diversity and an orthopraxic religious orientation (Japan).

It is our hope that the current research will inspire others to conduct replications and further examinations of religious priming effects, especially with understudied populations. Indeed, replications have been identified as a solution to the reproducibility crisis in the social sciences and may one day end debates within religious priming research [13,15,16].

Supporting information

Acknowledgments

We would like to thank Dhrumil Patel and Merve Akbas for advising on the Dots Game software. Jan Horský, Monika Bystroňová and František Brázda for being invaluable research assistants, and Eva Kundtová Klocová and HUME Lab Experimental Humanities Laboratory, Faculty of Arts, Masaryk University, for exceptional research support.