Figures
Abstract
At a fundamental level, taxonomy of behavior and behavioral tendencies can be described in terms of approach, avoid, or equivocate (i.e., neither approach nor avoid). While there are numerous theories of personality, temperament, and character, few seem to take advantage of parsimonious taxonomy. The present study sought to implement this taxonomy by creating a questionnaire based on a categorization of behavioral temperaments/tendencies first identified in Buddhist accounts over fifteen hundred years ago. Items were developed using historical and contemporary texts of the behavioral temperaments, described as “Greedy/Faithful”, “Aversive/Discerning”, and “Deluded/Speculative”. To both maintain this categorical typology and benefit from the advantageous properties of forced-choice response format (e.g., reduction of response biases), binary pairwise preferences for items were modeled using Latent Class Analysis (LCA). One sample (n1 = 394) was used to estimate the item parameters, and the second sample (n2 = 504) was used to classify the participants using the established parameters and cross-validate the classification against multiple other measures. The cross-validated measure exhibited good nomothetic span (construct-consistent relationships with related measures) that seemed to corroborate the ideas present in the original Buddhist source documents. The final 13-block questionnaire created from the best performing items (the Behavioral Tendencies Questionnaire or BTQ) is a psychometrically valid questionnaire that is historically consistent, based in behavioral tendencies, and promises practical and clinical utility particularly in settings that teach and study meditation practices such as Mindfulness Based Stress Reduction (MBSR).
Citation: Van Dam NT, Brown A, Mole TB, Davis JH, Britton WB, Brewer JA (2015) Development and Validation of the Behavioral Tendencies Questionnaire. PLoS ONE 10(11): e0140867. https://doi.org/10.1371/journal.pone.0140867
Editor: Martin Voracek, University of Vienna, AUSTRIA
Received: February 13, 2014; Accepted: October 1, 2015; Published: November 4, 2015
Copyright: © 2015 Van Dam et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Funding: This research was supported by the Kent Albert research fund at Yale University. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Even at the level of a single-celled organism, behavior must necessarily fall into one of three possible categories: move towards (approach), move away (withdraw) or neither (no response). These three basic options cannot be reduced further and therefore would seem to represent the most parsimonious description of behavioral tendencies. Human beings share an evolutionary heritage, which has disposed us for certain common proclivities [1]. Variations on these general proclivities, along with differences in our adaptations to considerably more recent challenges (in evolutionary terms) have led to relatively stable (both across individuals and cultures), and largely behavioral, inter-individual differences, identified as dispositional traits [2]. While these tendencies may vary considerably across individuals and cultures and may comprise complicated constructs such as personality (e.g., [3]), temperament, (e.g., [4]), and character (e.g., [5]), at a basic level, these constructs would seem to build upon a tripartite behavioral categorization of approach, avoid, and neither approach nor avoid (i.e., inaction).
The focus on these basic behavioral tendencies is not meant to undermine the laudable efforts at characterizing human personality, as many theories have resulted in impressive results. For example, relatively recent work has shown that the Big 5 (Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism), the most widely used categories of personality [3], show individual relationships to regional neuroanatomical volume [6] and intrinsic functional connectivity patterns of the brain [7]. However, the Big 5 may be characterized by two meta-traits, “Stability” and “Plasticity” [8], which may loosely relate to avoidance tendencies (stability—comprised of agreeableness, conscientiousness, and the inverse of neuroticism) and approach tendencies (plasticity—comprised of extraversion and openness).
One attempt to consider temperament/personality as tendencies for approach/avoidance was based in part on Gray’s neuropsychological model of anxiety [9]. Accordingly, Carver and White [10] developed a measure of the behavioral inhibition system (BIS) and the behavioral activation system (BAS), to which the fight-flight-freeze system (FFFS) has been appended in some conceptualizations [11]. While this early attempt to match personality to basic neurobehavioral tendencies is noteworthy, it is not without limitations. The original BIS/BAS [10] contained 13 items across 3 subscales related to appetitive motives (Drive, Fun Seeking, and Reward Responsiveness), but only 7 items on one subscale for avoidance. Further, the authors equated avoidance with behavioral inhibition (inaction) and fear, rather than the full range of behaviors commonly associated with the construct, including antipathy, repugnance, repulsion, dislike, hostility, anger, fear, and avoidance. Accordingly, the BIS items may reflect more fearful and neurotic tendencies than truly aversive ones, as reflected in strong correlations of the scale to negative affect, temperament, and harm avoidance [10]. Thus, while the BIS/BAS does reflect components of approach and avoidance, it may fail to capture essential pieces of avoidance. Further, it does not differentiate avoidance/aversion from the third fundamental option of inaction.
In addition to lacking basis in what could be considered the most parsimonious behavioral taxonomy (i.e., approach, avoidance, equivocation), most theories of temperament, personality, and character (e.g., [3–5, 12, 13]) are largely ‘Western’ in nature [3], potentially neglecting the cultural basis of meaning-making that forms the cornerstone of individual autobiographical representations [2]. Thus while existing models may meet aspects of criteria for one proposed evaluative system for testing personality theory (i.e., compatibility and predictive power); they may lack critical aspects of these criteria with regards to cross-cultural validity and parsimony [14, 15]. Further, because of their relatively recent conceptualization, current personality taxonomies have limited historical evidence to demonstrate their utility as a sustainably practical and cross-generational taxonomy tool.
Buddhist Approach to Temperament
Unlike most contemporary theories of personality, which are predominantly Western, and less than a century old [16], a Buddhist theory of temperament [17], outlined in the Visuddhimagga, a 5th century commentary on the early teachings of Theravada Buddhist doctrine, offers a non-western approach and one that has been in evidence for over fifteen hundred years (since at least the 5th Century C.E.). Although age itself does not necessarily translate to better classification, the consistent use of such a system over time and cultures (see e.g., [17–19]) suggests the possibility of cross-generational validity. Further, the traditional text from which this typology is drawn characterizes the system not only as predictive of numerous aspects of behavior (private and public), but also, primarily, as a means to prescribe lifestyle advice. These lifestyles are described as maximally effective for encouraging the cultivation of skillful habits and offsetting unskillful habits [19, 20], a practice that continues among modern-day practitioners of various types of meditation practices [19], as well as yoga and alternative medical practices among others [18, 21]. This classification system is still in use and has been adapted by different cultures across different contexts. Specifically, the successful translation of mindfulness practices described in the Visuddhimagga (“The Path of Purification”) to modern evidence-based interventions suggests that other concepts from the same textual sources may also be amenable to empirical evaluation.
The Buddhist Behavioral Temperaments
Buddhist theory posits that human beings react to pleasant experience by craving for its continuation, to unpleasant experience by craving for it to stop, and to neutral experience by ignoring it. Due to innate and learned dispositions, individuals are biased towards particular types of behavioral responses to stimuli. The Visuddhimagga refers to such temperaments using an Indic term, cariya, that translates literally as a ‘manner of going about’, or ‘manner of behavior’. This text presents its classification of behavioral types as a useful tool in tailoring interventions to individuals’ temperaments to achieve maximal benefit, specifying particular types of housing, clothing, and food, for instance, in addition to particular meditation techniques [17]. This may be one of the first descriptions of “personalized medicine”–matching an individual with a treatment based on the individual’s personal characteristics to improve outcomes.
The Buddhist classification of behavioral types considers numerous influences on how a person thinks, behaves, and feels, as well as basic organizational principles of the individual as an organism; the approach delineates these various influences in both external and internal factors. External signs include such factors as style of dress, postures and manner of walking, manner of eating, sleeping and waking, working style, and style of interacting with others. Internal signs mostly focus on the frequency of certain states of mind (e.g. anger, pride, etc.; more akin to many Western personality theories) [17, 19, 20].
The temperaments are characterized by three fundamental motivations (similar to those described above), greed (a pulling towards), aversion (a pushing away), and equivocation or confusion (characterized by neither pulling nor pushing but instead allowing the mind to wander either to disengage or escape from unsatisfactory conditions and/or to entertain tangential positive experiences related to satisfactory conditions). These three motivations correspond to three broad types of temperaments, which function as prototypes rather than exhaustive classifiers. In introducing the temperaments, the Visuddhimagga actually lists six types: greedy, faithful, aversive, discriminating, deluded, and speculative. However, the text suggests that the six types can be understood as forming three pairs of opposing properties corresponding to approach, avoid, and equivocate: 1) Greedy/Faithful type, the 2) Aversive/Discerning type, and the 3) Deluded/Speculative type. The two poles of each generally represent the skillful and unskillful aspects of each of these character types. For example, the greedy and faithful types, both manifest the same basic tendency, but the former in an unskillful way that leads to the perpetuation of and increase in suffering, whereas the latter manifests a skillful way that leads to the reduction of suffering ([17], p. 101–102). Each of these tendencies manifest in habitual ways, but importantly, also can be altered ([17], p. 102–3).
1. Greedy/Faithful temperament (Approach Oriented).
A style of seeking pleasant, attractive, optimistic, beautiful, and enjoyable things and experiences characterizes the Greedy/Faithful temperament. In general, individuals of this type tend to be passionate, extroverted, spontaneous, and seek out novel opportunities and experiences. With an attentional bias towards the positive, these individuals tend to overlook faults or dangers. They are biased away from seeing their own faults, for instance, and thus may tend toward personal vanity. Equally, such positive biases can lead to a neglect of possible negative consequences (e.g. ‘selling’ an idea without responsibly mentioning possible downsides). On the other hand, this trait can manifest in positive ways, inclining individuals to be generous with their trust and with material gifts, and to take visionary leaps of faith necessary to move good works forward, saying ‘yes!’ when more detail-oriented individuals might consider all the ways in which things could go wrong.
Greed has also been characterized as unwholesome craving or desire. Predominantly greedy individuals could be characterized by always wanting more. “It desires fulfillment through pleasures, finding what it likes in the world of the senses. From liking, it can move quickly to craving, passion and sensuality” ([19], p. 137). If greed is a destructive and unskillful emotion motivating the pursuit of sensual objects, the corresponding quality of “faith”, here, is a motivation to pursue what is skillful and virtuous, “…greed does not give up what is harmful, while faith does not give up what is beneficial” ([17], p.102). As Gethin [22] puts it, “…Buddhist texts understand faith…not so much as a matter of intellectual assent to certain propositions about the world…[but]as a state of trust, confidence, affection, and devotion…” (p. 167). Thus while benevolence or generosity might be seen as the more precise opposite of greed, here benevolence is only one example of a virtuous quality that is motivated by faith. While affable, and often well liked, individuals of the greedy/faithful type often have difficulty with discipline, restraint, and commitment, and can become vain, self-centered, and unreliable while looking for the “best offer”. They are vulnerable to overindulging and addictions, sexual infidelity, and deception [17, 19, 20]. For additional discussion of the characteristics of the greedy/faithful type, especially behavioral descriptors, see Table 1.
In suggesting how to tailor a therapeutic intervention to persons of the greedy type, the Visuddhimagga advises providing conditions that counteract the tendency to grasp after sensual pleasures. Thus it is suggested that providing luxurious housing, clothing, and food is not beneficial for one who is predominantly greedy, and moreover that it is helpful to have personal interactions that discourage pride and vanity ([17], p. 107–8). For similar reasons, meditative contemplation of the body is especially suggested as a meditation object for those of a greedy type, including contemplation of bodily anatomy such as organs and fluids ([17], p. 113). This seems to be intended to counter attachment to the body, others’ as well as one’s own, and aimed ultimately at fostering a more balanced mental attitude.
2. Aversive/Discerning temperament (Avoidance Oriented).
In contrast to the greedy type, the aversive type seeks to avoid or remove what is unpleasant, and therefore has an attentional bias towards faults, imperfections, burdens, and threats. Aversive types thrive in intellectual pursuits that require high levels of discernment, accuracy, and precision, and are often perceived as having high levels of competency or understanding. Where the greedy/faithful type is good at making the visionary leaps of faith necessary to move good works forward, aversive/discerning types excel at making detailed contingency plans to anticipate the many ways in which plans could go wrong. On a more negative note, however, these same qualities incline aversive types to judgment, criticism, hostility, hatred, prejudice, ill will, and aggression, all of which can lead to conflict and unpopularity. “The aversive temperament is constructed around judgment and rejection of experience…It is critical, quickly displeased, quarrelsome and disparaging of many things. Its quality of aversion can give rise to states of anger, vindictiveness, haughtiness, hatred, cruelty, aggression and struggle to control…” ([19], p. 174). Skillful and unskillful aspects are both recognized in the source text, which notes that whereas aversion is an unskillful and unwholesome motivation to find fault, discernment finds fault with those conditions that are not beneficial, in particular, unwholesome states of mind [17]. For additional discussion of the characteristics of the aversive/discerning type, especially behavioral descriptors, see Table 1.
In suggesting how to tailor a therapeutic intervention to persons of this type, the Visuddhimagga advises providing much more sensually pleasing conditions than those suggested for the greedy type. This text suggests aesthetically pleasing living quarters, with pleasing decorations, flowers, clothes, and perfumes, fine light clothing, as well as food that is inviting and “superior in every way” ([17], p. 108). The conditions suggested thus seem to be aimed at counteracting the tendency to aversion and of pushing away. Similarly, the development of loving kindness, compassion, sympathetic joy, and equanimity are especially recommended as meditation practices for one of an aversive temperament ([17], p. 114).
3. Deluded/Speculative temperament (Equivocate).
While the greedy/faithful and aversive/discerning types are characterized by their strong motivational tendency of either grasping or pushing away, the deluded/speculative type is distinguished by the absence of a strong motivational tendency. It is characterized by a lack of awareness of present conditions, whether pleasant, unpleasant, or neutral. As a result, deluded/speculative types tend not to have strong immediate reactions and not to hold fixed opinions. On the positive side, this allows for creativity, being open to many possible options, and ‘thinking outside the box’. On the negative side, the deluded type can seem lost, constantly scattered, and prone to following the opinions of others because of uncertainty. Without an agenda, they can be laidback and easy-going, creative and “out of the box” pioneers. However, the lack of clarity, goals, or direction can lead to confusion, doubt, paralyzing indecisiveness, speculation, and inaction. “They seek to establish ease by ignoring what is happening or through dullness or inaction. The deluded temperament gives rise to perplexity and worry, doubt, negligence, scattered thoughts, anxiety and agitation.” ([19], p. 175). For additional discussion of the confused/speculative type, especially behavioral descriptors, see Table 1.
Current Study
The current study aimed to develop a historically accurate and contemporarily applicable temperament questionnaire based on both traditional and contemporary texts, originating from the Visuddhimagga [17]. To counteract the effects of response styles and biases that can be particularly detrimental in cross-cultural research [23], we utilized a forced-choice response format, whereby items representing behaviors characteristic of the three types were compared. The psychometric properties of the questionnaire and specifically its ability to discriminate between the types were tested in two separate samples via Latent Class Analysis of choices made between pairs of items. Similar to Thurstonian Item Response Theory (IRT) modeling [24–27], this approach uses dummy-coded outcomes of rankings in each block of items for analysis. However, the measurement model assumes that categorical (latent classes) rather than continuous (latent traits) variables underlie the responses. To examine construct validity, we probed the relationship of the classes identified by this questionnaire to the predominant contemporary personality typology (i.e., the Big 5), as well as to approach/avoidance tendencies and behaviors, interpersonal tendencies, and decision-making styles.
Materials and Methods
Participants
A small sample of participants were recruited from the communities of Providence, RI and New Haven, CT (n = 36). All other participants (n = 859) were recruited through Amazon.com’s mechanical turk (MTURK) system. The MTURK works by creating a Human Intelligence Task (HIT), which are completed by an experimenter-selected number of potential participants from a pool of 500,000+ workers that use MTURK (see www.mturk.com). The experimenter creates an account to upload funds and create HITs. Reimbursement rate for a specified HIT is determined by the experimenter, often based on current norms among the MTURK for comparable amounts of time required to complete a HIT (see e.g., [28] for discussion of varying pay rates and its relatively small impact on the quality of data). Previous data analysis with MTURK has shown excellent internal consistency and good reliability, including high test-retest reliability [28]. Additionally, recent work has shown that data acquired via MTURK is not only equivalent to data collected in person, but also tends to provide more diverse demographics [29]. Data were collected in two waves. Data from the first wave were collected from a total of 394 individuals, 36 locally, and 354 from MTURK. Data from the second wave were collected from a total of 502 participants, all via MTURK. All procedures were determined to be exempt from institutional review board review due to the completely anonymous nature of the data collection.
Measures
Behavioral Tendencies Questionnaire (BTQ).
An interdisciplinary team (comprising contemplative Buddhist scholars and/or practitioners, philosophers, clinical, social, personality, and other psychologists, as well as neuroscientists and psychiatrists) was formed to discuss questionnaire development. Appropriate source documents were first identified to develop a comprehensive representation of each of the different temperament types. After becoming familiar with the various descriptions of each of the temperament types, thematic analysis was performed and preliminary summaries of the temperaments were created using contemporary and classic texts (e.g., [17, 19]). Source documents consistently listed eight behavioral contexts (i.e. sleeping, waking, social interactions, etc.; see Table 1 for details). Individuals from the team drafted items for each behavioral category for each temperament type and consensus building was used to compile questions upon which the group agreed. The initial items were piloted on a small group for content consistency.
After the initial pilot, a set of 43 blocks of three items remained, each containing a stem phrase and three different possible responses, corresponding to each of the three temperament types. The response format for the BTQ was multidimensional forced choice, wherein participants were instructed to rank response options (i.e., 1st, 2nd, 3rd) based on how well each option characterized them for a given stem. Participants were instructed to rank the response option that was most like them as 1st and the option that was least like them as 3rd in each block.
Big Five Aspect Scales.
The BFAS [12] is a 100-item measure of the ‘Big 5’ personality characteristics (openness, conscientiousness, extraversion, agreeableness, and neuroticism). Each ‘Big 5’ characteristic is comprised of two aspects: Openness = Intellect and Openness; Conscientiousness = Industriousness and Orderliness; Extraversion = Enthusiasm and Assertiveness; Agreeableness = Compassion and Politeness; Neuroticism = Withdrawal and Volatility. Each item is rated on a 1 (strongly disagree) to 5 (strongly agree) Likert-type scale. Past studies have revealed good psychometric properties and relationships to individual differences in brain structure [6, 12]. In the present sample, internal consistency for Big 5 (and domains) was as follows: Neuroticism, α = .906 (Withdrawal, α = .799; Volatility, α = .898); Agreeableness, α = .889 (Compassion, α = .897; Politeness, α = .776); Conscientiousness, α = .869 (Industriousness, α = .847; Orderliness, α = .811); Extraversion, α = .830 (Enthusiasm, α = .609; Assertiveness, α = .857); Openness, α = .869 (Openness, α = .839; Intellect, α = .814).
Behavioral Inhibition/Behavioral Activation Scales.
The BIS/BAS [10] is a 20-item questionnaire designed to assess behavioral approach and avoidance tendencies. The behavioral approach system (BAS) is assessed across 13 items, while the behavioral inhibition system (BIS) is assessed across 7 items. Participants rate each item on a 4-point Likert-type response scale ranging from 1 (very true for me) to 4 (very false for me). In order to facilitate easy interpretation alongside other measures, we reverse-scored the scale such that higher mean scores indicate higher approach or inhibition. The BIS/BAS has shown generally good psychometric properties, with a 4-factor structure (BIS + 3 BAS subscales), and reliability/consistency values mostly > .7 in both nonclinical [10] and clinical samples [30]. In the present sample, internal consistency was as follows: BAS, α = .860 (Drive, α = .844; Fun Seeking, α = .776; Reward Responsiveness, α = .779); BIS, α = .854.
Conner’s Adult ADHD Rating Scales.
The CAARS:S [31] is a 26-item, short-form assessment tool of Attention Deficit Hyperactivity Disorder symptoms. The items are rated on a 0 (not at all, never) to 3 (very much, very frequently), Likert-type response scale. The CAARS:S assesses four factor-derived subscales (Inattention and Memory Problems, Hyperactivity and Restlessness, Impulsivity and Emotional Lability, and Problems with Self-Concept), in addition to providing an inconsistency index. The inconsistency index consists of 16 items that are rated on a Likert-type scale ranging from 0 (not at all, never) to 3 (very much, frequently). The absolute difference for response pairs is summed and a cut-score is used. The index is an overall sum of absolute differences between pre-identified pairs of highly correlated items [31]. In the present sample, internal consistency was as follows: Inattention and Memory Problems, α = .840; Hyperactivity and Restlessness, α = .697; Impulsivity and Emotional Lability, α = .814; Problems with Self-Concept, α = .907.
Coping Responses Inventory.
The CRI [32] is a 48-item questionnaire designed to assess approach and avoidance coping strategies in response to dealing with problems. The scale consists of 24 items that assess approach coping (subscales include logical analysis, positive reappraisal, seeking guidance and support, and problem solving) and 24 items that assess avoidance coping (subscales include cognitive avoidance, acceptance/resignation, seeking alternative rewards, and emotional discharge). Individual items are rated on a 1 (not at all) to 4 (fairly often) Likert-type response scale. The CRI has shown good predictive validity of psychological problems (esp. problem drinking) [33]. In the present sample, internal consistency was as follows: Approach coping, α = .890 (Logical analysis, α = .701; Positive reappraisal, α = .846; Seeking guidance and support, α = .637; Problem solving, α = .766); Avoidance coping, α = .850 (Cognitive avoidance, α = .890; Acceptance or resignation, α = .818; Seeking alternative rewards, α = .704; Emotional discharge, α = .514).
Life Orientation Test—Revised.
The LOT-R [34] is a 6-item questionnaire (10 items if filler items are included) designed to measure optimism/pessimism. Three of the six items directly assess optimism and three directly assess pessimism. All items are scored so that higher scores indicate greater optimism. Individual items are rated on a 1 (I agree a lot) to 5 (I disagree a lot) Likert-type scale. In the present study, internal consistency was Cronbach’s α = .902.
Melbourne Decision Making Questionnaire.
The MDMQ [35] is a 22-item questionnaire designed to assess coping patterns associated with the conflict theory of decision making [36]. Items are rated on a 0 (not true for me) to 2 (true for me), 3-point Likert-type response scale. Factor analysis supported a 4-factor structure comprised of the constructs of Buck-Passing (i.e., encouraging others to make difficult decisions), Hypervigilance (i.e., a hurried, anxious style of decision making), Procrastination (i.e., delaying/avoiding making decisions), and Vigilance (i.e., an exhaustive search style of decision making). The MDMQ has shown good psychometric properties across international samples [35]. In the present sample, internal consistency was as follows: Buck Passing, α = .891; Hypervigilance, α = .785; Procrastination α = .877; Vigilance α = .805.
Trust Inventory.
The TI [37] is a 40-item measure of trust in others, generally (20 items) and romantic partners, specifically (20 items). Items are rated on a 1 (strongly disagree) to 4 (strongly agree), 4-point Likert-type rating scale. The scale has exhibited generally good psychometric properties [37]. Only the general trust items were administered in the present study. In the present study, internal consistency for the general subscale was, α = .708.
Modeling Forced-Choice Data
A forced-choice response format has several benefits, including elimination of response biases acting uniformly across all items such as extreme responding and central tendency responding, and acquiescence [38], as well as minimization of exaggerated emotional coherence in assessments of different qualities (halo effect) [39]. Additionally, forced-choice formats eliminate the need for a rating scale, which assumes that all participants interpret that response option categories in the same way—often a problematic assumption (see e.g., [25]). However, forced-ranking items present unique challenges to data analysis. The primary problem is that if one considers means or sums of the rankings (e.g., 1, 2, 3), the values will be the same for each block (i.e., M = 2, ∑ = 6), regardless of the actual ranking. The resulting data has been called ordinal ipsative data (e.g., [40]) and requires complex analytic approaches (e.g., [41, 42]), with difficult to interpret results.
Fortunately, a solution to modeling multidimensional force-choice questionnaire data has recently been proposed [24–27]. This method, identified as Thurstonian Item Response Theory (IRT) allows for the recovery of latent trait structures underlying a response set [24–27]. Thus, for each ranking block of three items, all pairwise comparisons of items (i.e., A vs. B, A vs. C, B vs. C) are considered. Each pairwise comparison (say, A vs. B) is coded {A, B} = 1 if item A was preferred to B, and {A, B} = 0 otherwise. Hence, the three comparisons/contrasts (binary dummy variables) fully describe any given rank order of three items. For example, if a respondent assigns the rank orders A = 2, B = 1, and C = 3, the dummy coding will be {A, B} = 0, {A, C} = 1, {B, C} = 1. Because questionnaire items are usually designed to indicate some underlying trait, each pairwise comparison indicates two traits. Here we assume that items within a block indicate different traits. Thus, comparisons {A, B} and {A, C} provide information about the latent trait associated with A, comparisons {A, B} and {B, C} provide information about the latent trait associated with B, and comparisons {A, C} and {B, C} provide information about the latent trait associated with C. Detailed information about the development of these models and their implementation can be found in previous publications [24–27].
However, the questionnaire in the present study was designed not to indicate latent traits (continua along which participants may be placed), but latent classes (nominal types into which participants may be placed). These types (greedy, aversive and deluded) were hypothesized to underlie the observed preferences; hence, the measurement model suitable for the questionnaire is not an IRT model, but a Latent Class Analysis (LCA) model. This model is logically straightforward: each binary dummy variable resulting from pairwise comparison of two items is underlain by a categorical latent variable (latent class). Specifically, outcomes resulting from comparisons between Greedy and Aversive items (the first and the second items in each BTQ block, see S1 Appendix) are underlain by a latent categorical variable taking two possible values, or two classes—Greedy and Aversive. Those respondents who belong to the Greedy class, should tend to prefer Greedy items, hence their response to all item-pairs of this kind should be primarily {iG, iA} = 1. Conversely, respondents belonging to the Aversive class, should demonstrate primarily responses {iG, iA} = 0. The other two types of item comparisons are modeled in the same way. Thus, to explain the patterns of choices between items indicating Greedy and Deluded (first and third items in each block), a latent categorical variable with two levels is needed. Finally, choices between Aversive and Deluded items (second and third items in each block) are explained by a latent categorical variable with two levels.
In this study, we tested the three separate LCA models described above, which have an advantage of simplicity at the initial stages of developing and testing the questionnaire. Conditional on the latent classes, the pairwise responses within each model are independent; hence, all assumptions of a LCA measurement model are met to explore the optimal number of classes needed to describe the variability in responses, and assess the performance of individual items in measuring the intended classes. Due to local dependencies occurring between pairwise comparisons involving the same item (for example, {A, B} and {A, C}), and associated modeling challenges, we do not test all pairwise comparisons simultaneously in this study.
The LCA analyses were performed in MPLUS v7.2 [43], using the maximum likelihood estimator (robust). All other analyses were conducted in SPSS v20.
Results
Demographics and Data Screening
Demographic comparisons.
Within sample 1, there was no age difference between participants recruited locally and those recruited via Amazon.com’s Mechnical Turk, t(382) = 1.40, p = .16. Similarly, there was no difference in gender distribution, χ2(1) = 0.12, p > .5, racial/ethnic background, χ2(5) = 2.74, p > .5, marital status, χ2(5) = 0.80, p > .5, or household income, χ2(8) = 10.40, p = .24. There were, however, significant differences in educational level, χ2(4) = 21.89, p < .001, and occupational status, χ2(5) = 18.21, p < .01. The local sample had higher rates of completed graduate/professional training (36.1% vs. 12.6%), college/university completion (52.8% vs. 43.0%), but lower rates of partial college (11.1% vs. 41.1%). The local sample had substantially lower rates of unemployment (0.0% vs. 19.3%) and part-time employment (5.6% vs. 21.5%), but higher rates of full-time employment (63.9% vs. 36.6%) and not in labor force (27.8% vs. 20.7%). The rates of inconsistent responding, based on item pairs from the BFAS, were no different between groups, suggesting combination of groups would not be inappropriate.
Exploratory LCA Analyses in Sample 1 Data
The dummy coded binary contrasts in each model were explored fitting a sequence of LCA models with 1, 2, 3 etc. classes in sample 1, which contained responses to the full BTQ (43 blocks). Bayesian Information Criterion (BIC) was used to determine the best number of classes [44]. Exploring the between-class odds ratios of preferring the first option in each item-pair to the second option in calibration sample (sample 1) helped determine well performing items (items that discriminate well between the two classes) and flag poorly performing items. To this end, we used the recommended effect sizes for odds ratios (small = 1.5, medium = 3.5 and large = 9 [45]).
Greedy versus aversive.
Dummy coded binary contrasts between the 1st and the 2nd item in each block were explored fitting a sequence of LCA models with 1, 2, 3 etc. classes. Table 3 reports the BIC and the entropy (index of classification quality) for each of the competing models. It can be seen that the 2-class solution was the best. The two classes were very clearly interpretable—the conditional probabilities of preferring the first item to the second in each pair were always higher in the first class. Hence, the first class represented the Greedy type (57% estimated prevalence), and the second represented Aversive (43% prevalence). Exploring the between-class odds ratios, two blocks showed large effect sizes, 12 blocks showed medium to large effect sizes, and 21 small to medium. Eight blocks discriminated poorly with effect sizes less than 1.5 (small).
Greedy versus deluded.
Dummy coded binary contrasts between the 1st and 3rd item in each block were explored fitting a sequence of LCA models. From Table 3 it can be seen that the 2-class solution was the best. The two classes were clearly interpretable as the Greedy type (58.2% estimated prevalence), and the Deluded type (41.8% prevalence). Exploring the between-class odds ratios, there were no large effects, 13 blocks showed medium to large effect sizes and 19 small to medium. Eleven blocks discriminated poorly with effect sizes less than 1.5 (small).
Aversive versus deluded.
Dummy coded binary contrasts between the 2nd and 3rd item in each block were explored fitting a sequence of LCA models. From Table 3 it can be seen that the 2-class solution was again the best. The two classes were clearly interpretable as the Aversive (45.2% estimated prevalence) and Deluded types (54.8% prevalence). Exploring the between-class odds ratios, three blocks showed large effect sizes, 11 blocks showed medium to large effect sizes, and 14 small to medium. Fifteen blocks discriminated poorly with effect sizes less than 1.5 (small).
Classification of Participants in Sample 2
After the item-pair parameters (i.e. thresholds) conditional on latent class membership have been established using sample 1, these parameters were used to calculate the conditional probabilities of belonging to latent classes for sample 2. Because sample 2 was administered a reduced version of the BTQ (with 29 blocks), only parameters for the available blocks were used. This calculation led to obtaining for each participant:
- the probability of being Greedy from the model contrasting Greedy and Aversive, PGA(G), and the complementary probability of being Aversive PGA(A) = 1 –PGA(G);
- the probability of being Greedy from the model contrasting Greedy and Deluded, PGD(G), and the complementary probability of being Deluded PGD(D) = 1 –PGD(G); and
- the probability of being Aversive from the model contrasting Aversive and Deluded, PAD(A), and the complementary probability of being Deluded PAD(D) = 1 –PAD(A).
Thus, membership in each temperament class was based on the evidence from two separate models. Having these multiple evidences was beneficial since it allowed determining not only primary temperament/tendency type, but also secondary type. For instance, if an individual has the probabilities PGA(G)>PGA(A), PGD(G)>PGD(D) and PAD(A)>PAD(D), it must follow that his/her primary type is Greedy (since it was the most likely type compared to the alternatives), and his/her secondary type is Aversive. Given 3 classifications with 2 classes each, theoretically there should be 23 = 8 perturbations of the probability inequalities. However, two perturbations out of eight are not admissible since they represent intransitive inequalities: (1) PGA(G)>PGA(A), PGD(G)<PGD(D) and PAD(A)>PAD(D) and (2) PGA(G)<PGA(A), PGD(G)>PGD(D) and PAD(A)<PAD(D). The remaining six inequalities are transitive and result in the following primary and secondary type combinations: GA, GD, AG, AD, DG, and DA. Considering only the primary type classification, out of all valid cases from sample 2, 42% were classified as the Greedy type, 32.6% as the Aversive type and 25.4% as the Deluded type. The primary type attributions were then used to validate the BTQ against other measures used in this study.
Construct Validation with Sample 2
Sample 1: Relationships to Big Five.
After classification of each participant into a primary temperament type based on the BTQ, the three groups were compared on scales and subscales of the BFAS. ANOVAs with Bonferroni corrected post-hoc tests were conducted on all 10 aspect scales and the Big Five scales. Group differences are reported in Table 4.
The greedy/faithful types were (relatively) highest on compassion (aspect scale of agreeableness) and extraversion (enthusiasm, and assertiveness, relative to the deluded/speculative type). The aversive/discerning types were highest on conscientiousness (overall, and orderliness aspect) and lowest on openness (aspect of Openness trait). Finally, the deluded/speculative types were higher than the greedy/faithful types (but not aversive/discerning types) on neuroticism (withdrawal and volatility aspects, as well as neuroticism trait) and lowest on conscientiousness (industriousness and orderliness aspects, as well as conscientiousness trait).
Sample 2: Relationships to Approach/Avoidance, Attentional Functioning, Interpersonal functioning, and decision making.
Group differences were largely consistent with hypotheses (see Table 5). The greedy/faithful types were highest on overall approach orientation (BAS), and the drive and reward-responsiveness subscales of the BAS, the overall approach scale of the CRI, and positive reappraisal, and seeking guidance subscales of the CRI, the LOT-R (optimism), and the Trust Inventory. The aversive/discerning types were lowest on overall approach orientation (BAS) and the fun-seeking subscale of the BAS, lowest on the overall avoidance scale of the CRI, and significantly higher than the deluded/speculative types (but not the greedy/faithful types) on the vigilance subscale of the MDMQ. The deluded/speculative types were highest on all aspects of the CAARS (inattention and memory problems, hyperactivity and restlessness, impulsivity and emotional lability, and self-concept problems) as well as on the inconsistency index. They were also higher on cognitive avoidance, acceptance and resignation, and the emotional discharge subscales of the CRI, along with the overall avoidance scale of the CRI, as well as lowest on the overall approach scale of the CRI, and the logical analysis and problem-solving subscales of the CRI. The deluded/speculative types were additionally lowest on the LOT-R and highest on the Buck-Passing, Hypervigilance, and Procrastination subscales of the MDMQ.
Development of the BTQ short form
Based on the odds ratios computed for all pairs in sample 1 (odds of preferring the first item in the pair for members of class 1 divided by odds of preferring the first item in the pair for members of class 2), we selected the best discriminating blocks from the original 43 blocks (13 in total) to create a short version of the BTQ (BTQ-SF). The BTQ-SF is provided in S1 Appendix. Conditional probabilities of preferring the first item in each pair are given in Table 6. Using conditional probabilities, the individual probabilities of belonging to each class may be computed, and individuals classified in one of the three classes. We compared the estimated class membership based on the full BTQ (43 blocks) and the short version (13 blocks). The shortened version yielded very similar classification (see Table 7), with the coefficient of agreement between the two classifications kappa = .798.
Discussion
The goals of the current project were to establish a parsimonious temperament scale based on historic observations and considerations of habitual behaviors [17] that had 1) good psychometric properties, and 2) good nomothetic span (appropriate relationships to other self-report measures).
We purposefully retained the original forced-choice format, ranking 3 response options with a single item stem, corresponding to each of the behavioral tendencies. Retaining this format was important, as the items were designed to represent categorical options broadly relating to descriptions of what individuals of different types would do in different contexts [17, 19]. Latent Class Analysis of pairwise comparisons between items indicating Greedy and Aversive types yielded two classes, which clearly corresponded to the hypothesized types. The same was true for the contrasts between Greedy and Deluded, and between Aversive and Deluded types. Individual items were assessed for their ability to discriminate between the types; many items had favorable properties, showing medium to large or large effect sizes. While we created an item pool reflecting the many characteristics of the traditional temperaments (see Table 1), not all item domains had acceptable psychometric properties. For example, items about food preferences had poor discriminative ability. Overall, however, the items of the final scale (BTQ-SF, see S1 Appendix) generally reflected the broad characteristics related to these historic temperament types. The class membership derived from the estimated conditional probabilities of responses resulted in clear classification with one predominant behavioral tendency type, as well as a secondary type.
When examining the classification groups of the first sample in relation to the Big Five Aspect Scales, the Greedy/Faithful types were compassionate and extraverted, the Aversive/Discerning types were conscientious and closed-minded (opposite of openness to experience), and the Deluded/Speculative were somewhat neurotic and careless (opposite of conscientious). The second sample completed measures of approach/avoidance (BIS/BAS), attentional problems (CAARS:S), coping styles (CRI), optimism, decision-making styles, and trust. As hypothesized, the Greedy/Faithful types were the most Approach-oriented, with significantly higher BAS scores than both other types on 2/3 approach-oriented subscales and the total BAS score. The Aversive/Discerning types had the lowest approach orientation overall (BAS), while the Deluded/Speculative types had the highest behavioral inhibition (BIS) mean score. Given that the BIS measures behavioral inhibition (more akin to inaction and/or fear) rather than avoidance [10], it is perhaps not surprising that the Deluded/Speculative types were highest on this measure. While the CAARS:S was designed to assess symptoms of ADHD, the nature of subscales (inattention and memory problems, hyperactivity and restlessness, impulsivity and emotional lability, and problems with self-concept) is consistent with the Deluded/Speculative type, who exhibited the highest scale means for all these subscales. Of additional interest, the Deluded/Speculative types exhibited the highest mean inconsistency scores, which provides support for this typology as a behavioral indicator of equivocation and/or difficulty committing to a given response (pattern).
In terms of coping strategies, the Greedy/Faithful types were largely approach-oriented in terms of problem-solving while the Deluded/Speculative types were largely avoidance-oriented. While we characterized the Aversive/Discerning types as generally avoidance-oriented, these individuals are also generally logical and conscientious (see Table 1), thus it is reasonable to conclude that they may not necessarily avoid problems while the indecisive nature of the Deluded/Speculative types may lead them to avoid problems in a number of ways.
In terms of optimism, the Greedy/Faithful types were highest and the Deluded/Speculative types lowest. While it is hypothesis-consistent that the Greedy/Faithful types were highest, one might have expected the Aversive/Discerning types to be lowest on optimism. However, one possibility is that the general uncertainty and ambiguity with which the Deluded/Speculative types view the world that leads them to be somewhat more pessimistic. Consistent with our hypotheses, the Deluded/Speculative type had the highest levels of maladaptive decision-making, exhibiting high levels of buck-passing, hypervigilance, and procrastination. Also consistent with our hypotheses, the Greedy/Faithful types had the highest mean scores on the Trust Inventory. Overall, it would seem that the types derived from the BTQ would seem to be largely consistent with those identified by the early source text [17].
Limitations
It is worth noting that the temperaments, as originally described, rely heavily on observable aspects of behavior [17]. Historical and contemporary assignments of a particular temperament type were often done by experienced individuals who had detailed (longitudinal) knowledge of the individuals they were categorizing [20]. The present study relied solely on self-report of behavioral tendencies, which may pose unique problems for a typology historically characterized by observable behaviors. Items which held up to psychometric analysis in the present study may not represent aspects of the constructs that are better measured by observer report, actual behavioral measure, etc.
Future Directions
Similarities to attachment types.
The three insecure types of attachment [46] map relatively well with the behavioral temperaments. The anxious attachment style is one of clinging and neediness, as well as concern about involvement with others, similar to the Greedy temperament. The avoidant attachment style is one of pushing others away, often with increased internally and externally directed anger, and efforts to be self-sustaining, similar to the Aversive temperament. Finally, the disorganized attachment style is one of volatility and distortion, lacking a clear pattern or basis, similar to the Deluded temperament. Future studies might examine whether different attachment styles (e.g., [47]) map onto the behavioral temperaments. Relationships between these two constructs would further establish construct validity and compatibility and might permit important elaboration of how an individual’s attachment style might influence his/her behavioral predispositions.
Similarities to theoretical neurobehavioral systems.
Another important future project may be to examine relationships between the behavioral temperaments and the approach/avoidance systems as measured experimentally (e.g., [48]). The greedy/faithful type might be described as very approach-oriented, while the aversive/discerning type might be described as very avoidance oriented. The deluded/speculative type might be considered neither approach nor avoidance oriented as this type is fundamentally confused about whether to pull towards or push away. Accordingly, there may be interesting relationships between the behavioral temperaments and related neurobehavioral functions of approach and avoidance systems [49].
Similarities to other measures of temperament.
Other temperament scales have been developed with the aim of characterizing human experience and proclivities. For example, the Adult Temperament Questionnaire (ATQ) has shown similarities to the Big 5 in that it both had the emergence of a 5 factor model and subsequently demonstrated a higher-order 2 factor model [4, 50]. The Temperament and Character Inventory is another measure that has proposed relationships to basic neurochemistry [5]. Given that the behavioral temperament questionnaire has potentially important relationships to other measures of temperament (especially the ATQ; see e.g., [50]), future studies might examine these possible overlaps. Since the ATQ has direct relationships to various attentional and self-regulatory systems, these findings could potentially elucidate underlying neurocognitive components of the behavioral temperaments. Relationships of the BTQ to these scales might provide further insight into the construct validity of the BTQ, as well as helping to generate hypotheses about the neurophysiological basis of basic habitual behaviors.
Clinical and practical utility.
Finally, measures like the Behavioral Tendency Questionnaire may provide a much-needed framework to tailor behavioral approaches to treatment. Indeed, the text from which we have drawn the description of these types presents the typology primarily as a means to provide prescriptive practices in relation to each of the different temperaments. This suggests that recent initiatives among medicine to individualize treatment (e.g., [51]) can also be utilized in meditation training as well as other behavioral therapies. The behavioral temperament scale presented here may be useful for indicating who might respond best to what type of meditation techniques [19], body-based practices and lifestyle changes [18, 21]. We have noted, for instance, how the Visuddhimagga suggests that individuals who manifest behavioral characteristic of the aversive type will do best meditating in visually pleasing, beautiful settings, and should be given meditation practices such as developing loving kindness. In contrast, it is suggested that greedy types will find austere and even unpleasant conditions more conducive to practice, presumably because these counteract the tendency to focus on pleasant stimuli. Similarly, because of its function to cut off mind-wandering, mindfulness of breathing is suggested as particularly suitable for those with deluded and speculative tendencies [17] (p. 114).
Notably, this characterization of temperament/personality is quite different from the most commonly used approach (i.e., the Big 5). A great deal of psychopathology is associated with Neuroticism. While perhaps unintended, the classification of one as predominantly neurotic does not really permit much room for positive interpretation. Further, the proposed notion that these personality traits are stable over time implies that the neurotic individual may always be that way. In stark contrast, the behavioral temperament system promotes a positive and negative aspect of each temperament that can be modified through training. The greedy individual is also characterized as faithful, the aversive as discerning, and the deluded as speculative. These reflect skillful and unskillful manifestations of the same types of behaviors. Thus an individual characterized as one type can actively work to direct behaviors in the more skillful direction. Further, the nature of these simple classifications may serve to enhance an individual’s awareness of his/her oft-overlooked behavioral tendencies and serve as a basis for discussion. As the typology is relatively simplistic, recalling that one is predominantly of a particular type could help one to place effort towards being more skillful, especially in the company of others who may have known similar or dissimilar temperaments.
These types of individualized approaches may be relatively easily adopted in modern medicine, given the brevity and ease of use of screening tools such as the BTQ. Indeed, although the Visuddhimagga’s discussion envisions a teacher having ample time to observe a student’s behavioral habits in daily life, s/he recommends questioning the student as a primary means of determining temperament [17] (p. 107). Moreover, given the origins of the scale, the BTQ may offer high cultural acceptability and relevance to the increasing population of Buddhists in Western societies. In the modern clinical context, individuals might fill out a screening BTQ form before beginning practice with a teacher or taking a mindfulness-based clinical intervention (e.g. Mindfulness-Based Stress Reduction; [52]), allowing the teacher to tailor instruction accordingly. Using this empirically derived approach to tailoring treatments or practices to individual temperaments may improve the efficacy of current treatments, though specific clinical trials are warranted to test such hypotheses.
Conclusions
The present study aimed to develop and validate a behavioral tendencies questionnaire based on habitual behavior that was true to the traditional Buddhist texts in which mindfulness practice is described, has present day validity and applicability, and that seems to have practical utility in both scientific and clinical settings. The findings suggested good psychometric properties and construct consistent nomothetic span. Given the strong Buddhist influence on the BTQ, the measure would seem to have potential implications for individualizing meditation and mindfulness-based practices. Additional work is required to identify the extent to which the BTQ is a useful predictor of observable behavior, task performance, and neurobehavioral activity, but the present findings suggest that the scale has considerable potential.
Supporting Information
S1 Appendix. Behavioral Tendency Questionnaire.
https://doi.org/10.1371/journal.pone.0140867.s001
(DOC)
Acknowledgments
The authors would like to thank Dave Perlman, Zach Schlosser, Jeremy Gray, and Colin DeYoung for insightful comments and discussion that contributed to some of the ideas presented in this manuscript.
Author Contributions
Conceived and designed the experiments: NTVD TBM JHD WBB JAB. Performed the experiments: NTVD JAB. Analyzed the data: NTVD AB. Wrote the paper: NTVD AB TBM JHD WBB JAB.
References
- 1. Buss DM. Evolutionary Personality Psychology. Annual Review of Psychology. 1991;42(1):459–91. pmid:2018400.
- 2. McAdams DP, Pals JL. A new Big Five: Fundamental principles for an integrative science of personality. American Psychologist. 2006;61(3):204–17. pmid:16594837
- 3.
John OP, Naumann LP, Soto CJ. Paradigm shift to the integrative Big Five trait taxonomy: History, measurement, and conceptual issue. In: John OP, Robbins RW, Pervin LA, editors. Handbook of personality: Theory and research. New York: Guiford Press; 2008. p. 114–58.
- 4. Evans DE, Rothbart MK. Developing a model for adult temperament. Journal of Research in Personality. 2007;41:868–88. Epub 01/18/2007.
- 5.
Cloninger CR, Przybeck TR, Svrakic DM, Wetzel RD. The temperament and character inventory (TCI): A guide to its development and use. St. Louis, MO: Center for Psychobiology of Personality; 1994.
- 6. DeYoung CG, Hirsh JB, Shane MS, Papademetris X, Rajeevan N, Gray JR. Testing predictions from personality neuroscience. Brain structure and the big five. Psychological Science. 2010;21(6):820–8. Epub 2010/05/04. pmid:20435951; PubMed Central PMCID: PMC3049165.
- 7. Adelstein JS, Shehzad Z, Mennes M, DeYoung CG, Zuo X-N, Kelly C, et al. Personality Is Reflected in the Brain's Intrinsic Functional Architecture. PLOS ONE. 2011;6(11):e27633. pmid:22140453
- 8. DeYoung CG. Higher-order factors of the Big Five in a multi-informant sample. Journal of Personality and Social Psychology. 2006;91(6):1138–51. Epub 2006/12/06. pmid:17144770.
- 9.
Gray JA, McNaughton N. The neuropsychology of anxiety: An enquiry into the functions of the septo-hippocampal system. 2 ed. New York: Oxford Unversity Press; 2000.
- 10. Carver CS, White TL. Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS Scales. J Pers Soc Psychol. 1994;67(2):319–33. 1995-00067-001.
- 11. Corr PJ. Automatic and controlled processes in behavioural control: Implications for personality psychology. European Journal of Personality. 2010;24(5):376–403.
- 12. DeYoung CG, Quilty LC, Peterson JB. Between facets and domains: 10 aspects of the Big Five. Journal of Personality and Social Psychology. 2007;93(5):880–96. Epub 2007/11/07. pmid:17983306.
- 13.
Thomas A, Chess S. Temperament and development. Oxford, England: Brunner/Mazel 1977.
- 14. Smits IA, Dolan CV, Vorst HC, Wicherts JM, Timmerman ME. Cohort differences in Big Five personality factors over a period of 25 years. J Pers Soc Psychol. 2011;100(6):1124–38. pmid:21534699.
- 15. Specht J, Egloff B, Schmukle SC. Stability and change of personality across the life course: the impact of age and major life events on mean-level and rank-order stability of the Big Five. J Pers Soc Psychol. 2011;101(4):862–82. pmid:21859226.
- 16. Digman JM. Personality Structure: Emergence of the Five-Factor Model. Annual Review of Psychology. 1990;41(1):417–40.
- 17.
Buddhaghosa B. The Path of Purification: Visuddhimagga. Onalaska, WA: Pariyatti Publishing; 430 CE/2003.
- 18. Frawley D. Yoga and Ayurveda: A Call for a Complete Yogic System of Medicine. International Journal of Yoga Therapy. 2008;18(1):21–3.
- 19.
Kornfield J. The wise heart: A guide to the universal teachings of Buddhist psychology. New York: Bantam Books; 2008.
- 20.
Mann R, Youd R. Buddhist Character Analysis. Bradford on Avon, Wiltshire: Aukana Trust; 1992. 130 p.
- 21.
Busia K, editor. Iyengar: The yoga master. Boston, MA: Shambhala Publications; 2007.
- 22.
Gethin R. The foundations of Buddhism: Oxford University Press; 1998.
- 23. van Herk H, Poortinga YH, Verhallen TMM. Response Styles in Rating Scales: Evidence of Method Bias in Data From Six EU Countries. Journal of Cross-Cultural Psychology. 2004;35(3):346–60.
- 24. Maydeu-Olivares A, Brown A. Item Response Modeling of Paired Comparison and Ranking Data. Multivariate Behavioral Research. 2010;45(6):935–74.
- 25. Brown A, Maydeu-Olivares A. How IRT can solve problems of ipsative data in forced-choice questionnaires. Psychol Methods. 2013;18(1):36–52. pmid:23148475
- 26. Brown A, Maydeu-Olivares A. Fitting a Thurstonian IRT model to forced-choice data using Mplus. Behav Res Methods. 2012;44(4):1135–47. pmid:22733226.
- 27. Brown A, Maydeu-Olivares A. Item Response Modeling of Forced-Choice Questionnaires. Educational and Psychological Measurement. 2011;71(3):460–502.
- 28. Buhrmester M, Kwang T, Gosling SD. Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data? Perspectives on Psychological Science. 2011;6(1):3–5. pmid:26162106
- 29. Casler K, Bickel L, Hackett E. Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior. 2013;29(6):2156–60.
- 30. Campbell-Sills L, Liverant GI, Brown TA. Psychometric evaluation of the behavioral inhibition/behavioral activation scales in a large sample of outpatients with anxiety and mood disorders. Psychol Assess. 2004;16(3):244–54. pmid:15456380.
- 31.
Conners CK, Erhardt D, Sparrow E. Conners' adult ADHD rating scales: Technical manual. North Tonawanda, NY: Multi-Health Systems, Inc; 1999.
- 32.
Moos RH. Coping Responses Inventory: Professional Manual. Odessa, FL: Psychological Assessment Resources, Inc.; 1993.
- 33. Moos RH, Brennan PL, Fondacaro MR, Moos BS. Approach and avoidance coping responses among older problem and nonproblem drinkers. Psychology and aging. 1990;5(1):31–40. pmid:2180432.
- 34. Scheier MF, Carver CS, Bridges MW. Distinguishing optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): a reevaluation of the Life Orientation Test. J Pers Soc Psychol. 1994;67(6):1063–78. pmid:7815302.
- 35. Mann L, Burnett P, Radford M, Ford S. The Melbourne decision making questionnaire: an instrument for measuring patterns for coping with decisional conflict. Journal of Behavioral Decision Making. 1997;10(1):1–19.
- 36.
Janis IL, Mann L. Decision making: A psychological analysis of conflict, choice, and commitment. New York: Free Press; 1977.
- 37. Couch LL, Adams JM, Jones WH. The Assessment of Trust Orientation. Journal of Personality Assessment. 1996;67(2):305–23. pmid:16367682
- 38. Cheung MWL, Chan W. Reducing Uniform Response Bias With Ipsative Measurement in Multiple-Group Confirmatory Factor Analysis. Structural Equation Modeling: A Multidisciplinary Journal. 2002;9(1):55–77.
- 39. Bartram D. Increasing Validity with Forced-Choice Criterion Measurement Formats. International Journal of Selection and Assessment. 2007;15(3):263–72.
- 40. Chan W, Bentler P. Covariance structure analysis of ordinal ipsative data. Psychometrika. 1998;63(4):369–99.
- 41. Chan W. Analyzing ipsative data in psychological research. Behaviormetrika. 2003;30(1):99–121.
- 42. Jackson DJ, Alwin DF. The Factor Analysis of Ipsative Measures. Sociological Methods & Research. 1980;9(2):218–38.
- 43.
Muthen LK, Muthen BO. Mplus User's Guide. 6 ed. Los Angeles, CA: Muthen & Muthen; 1998–2010.
- 44. Nylund KL, Asparouhov T, Muthén BO. Deciding on the Number of Classes in Latent Class Analysis and Growth Mixture Modeling: A Monte Carlo Simulation Study. Structural Equation Modeling: A Multidisciplinary Journal. 2007;14(4):535–69.
- 45. Haddock CK, Rindskopf D, Shadish WR. Using odds ratios as effect sizes for meta-analysis of dichotomous data: A primer on methods and issues. Psychological Methods. 1998;3(3):339–53.
- 46. Ainsworth MD, Bell SM. Attachment, exploration, and separation: illustrated by the behavior of one-year-olds in a strange situation. Child Dev. 1970;41(1):49–67. pmid:5490680.
- 47. Fraley RC, Waller NG, Brennan KA. An item response theory analysis of self-report measures of adult attachment. J Pers Soc Psychol. 2000;78(2):350–65. pmid:10707340.
- 48. Gray JR. Emotional modulation of cognitive control: approach-withdrawal states double-dissociate spatial from verbal two-back task performance. J Exp Psychol Gen. 2001;130(3):436–52. pmid:11561919.
- 49. Amodio DM, Master SL, Yee CM, Taylor SE. Neurocognitive components of the behavioral inhibition and activation systems: Implications for theories of self-regulation. Psychophysiology. 2008;45(1):11–9. ISI:000252592700002. pmid:17910730
- 50. Evans DE, Rothbart MK. A two-factor model of temperament. Personality and Individual Differences. 2009;47(6):565–70. pmid:20161172
- 51. Norcross JC, Wampold BE. What works for whom: Tailoring psychotherapy to the person. Journal of Clinical Psychology. 2011;67(2):127–32. pmid:21108312
- 52.
Kabat-Zinn J. Full Catastrophe Living: Using the Wisdom of Your Body and Mind to Face Stress, Pain, and Illness. New York, NY: Random House, Inc.; 1990.