http://orcid.org/0000-0001-5929-4224
Figures
Abstract
Introduction
School victimization by peers is an important social problem with serious short- and long-term consequences poorly studied at preschool ages, which can lead to school bullying without timely intervention. Longitudinal data was used to determine the prevalence of warning signs of preschool peer victimization and its individual and family risk factors.
Methods
Data was obtained from 577 community preschoolers. School victimization was measured using the Strengths and Difficulties Questionnaire (SDQ) administered to parents and teachers of children at ages 4 and 5. Risk factors for the child (demographics, conduct and emotional problems, aggressiveness) and the family (maternal problems during pregnancy and early development, parenting styles, adaptive functioning and parents’ problems) were previously recorded at 3 years old.
Results
Combined information from parents and teachers showed that 4.2% of preschoolers presented warning signs of victimization at ages 4 and 5. Low socioeconomic status, poor emotional control, early problems making friends and low level of parenting education in social norms at age 3 predicted later victimization at ages 4 and 5 (AUC = .78).
Citation: Navarro J-B, Fernández M, de la Osa N, Penelo E, Ezpeleta L (2019) Warning signs of preschool victimization using the strengths and difficulties questionnaire: Prevalence and individual and family risk factors. PLoS ONE 14(8): e0221580. https://doi.org/10.1371/journal.pone.0221580
Editor: Kenji Hashimoto, Chiba Daigaku, JAPAN
Received: April 21, 2019; Accepted: August 11, 2019; Published: August 23, 2019
Copyright: © 2019 Navarro et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be made publicly available due to ethical restrictions protecting the confidentiality of the families involved (Ethics Commission of Animal and Human Experimentation of the Universitat Autònoma de Barcelona). Data are available to interested researchers after signing a consent of confidentiality form as the authors had previously signed to obtain the data from the families of the sample. Researchers must be working in clinical child psychology in a public funded project. To request the anonymous data, please contact the Department of Clinical and Health Psychology of the Universitat Autònoma de Barcelona (d.psic.salut@uab.cat).
Funding: This work was supported by the Spanish Ministry of Economy and Competitiveness (MINECO/FEDER) [grant PSI2012-32695] and by the Spanish Ministry of Science, Innovation and Universities (MICINN/FEDER) [grant PGC2018-095239-B-I00].
Competing interests: The authors have declared that no competing interests exist.
Introduction
Peer victimization is a developmentally salient interpersonal stressor that refers to the experience of being the recipient of peer intimidation and harassment or being the target of physical, social, emotional or psychological harm from a peer [1]. As a form of aggressive behavior there is the risk of escalation and if the victimization is repeated and intentional and there exists an imbalance of power between aggressors and victim, it takes the form of bullying [2]. School victimization is an important social problem with serious short-term consequences for victims’ physical and psychological health and negative long-term effects on their future psychosocial adjustment as adults.
The cumulative consequences of early victimization may emerge at later points in children’s school careers and have been well documented [3,4]. In cases where the victimization becomes chronic, the consequences may have an even greater effect on children’s achievements and learning [5]. Peer victimization has been associated with emotional and behavioral problems over time that are beyond the child’s pre-existing difficulties [6], with children who are victimized at risk of a wide range of adjustment difficulties that include: self-harm [7], loneliness, social avoidance, self-blame [8], suicide ideation [9], anxiety and depression [10], social maladjustment, loneliness [11], lack of close peer relationships [12], low self-esteem [13], poor physical health, psychosomatic complaints, behavioral dificulties and deficient emotional adjustment [13,14]. Other important findings show there is also a potential significative impact on adult adjustment outcomes [15] such as problems with doing housework and managing money [16], trouble forming new social relationships, less social support, poorer family functioning and lower levels of education [17].
Early diagnosis and timely treatment of these problems at a young age will not only improve children’s adaptation at school and their emotional and social development [18], but will also help to prevent their impact in later years.
Preschool and peer victimization
Most of the research about peer victimization has focused on middle childhood or adolescence. Beyond the home, school is usually the main environment where children’s difficulties with social interactions with their peers can be primarily detected and subsequently assessed by adults and professionals [18]. During preschool and kindergarten, children learn how to build and maintain friendships, and they form opinions about who they like or dislike and they establish groups of consistent play partners [19]. Experts on child development strongly agree on the importance of acquiring cognitive and social-emotional competencies at the pre-school stage [20].
The manifestation of peer victimization in the preschool years is similar to that of school-age children in many ways but differs in others depending on the child’s development of social cognition [21]. Younger students usually tend to resort to physical aggression, possibly because they have not yet developed the required sophistication of verbal or social skills to obtain what they want. With maturation, children start utilizing verbal skills to engage in more subtle forms of verbal aggression [22,23] and/or develop more complex social skills such as assessing and manipulating social situations, and they can also engage in indirect forms of aggression [22]. For these reasons it is important to study peer victimization from preschool ages.
Prevalence and measure
Although the most severe forms of peer victimization seem to have reduced in many countries over the last few years [24], efforts to detect and prevent this type of aggressive behavior in schools are still essential. There is an abundance of large-scale studies on elementary and high school children [12,25], but less studies have been carried out on preschoolers. The literature shows that even at these early ages peer victimization occurs frequently and is associated with poor psychosocial well-being [26]. Ilola et al. [27] found a 4% six-month prevalence of frequently bullied children as reported by the parents of a sample of 931 4-year-olds who attended pediatric checkups. In another large population-based study of 5- to 6-year-olds that used reports from parents and teachers, a similar 4% of the children were victims [10].
A problem like peer victimization, which remains hidden in most cases, requires information from as many sources as possible. The Strengths and Difficulties Questionnaire (SDQ) [28] is a tool with good psychometric properties that is widely used to screen for psychiatric disorders and its version for children aged 3 to 4 years old has been utilized at preschool age [29,30]. The SDQ has versions for parents and teachers to account for discrepancies between informants, since different informants observe children’s behavior in different settings [31]. Parents and teachers are the ones who spend most time with pre-school aged children. Given the preventive aim of early detection, both reports are indispensable under the Attribution Bias Context Model [32], which posits that informant discrepancies are indicative of cross-contextual variability in children’s behavior and informants’ perspectives on this behavior. The SDQ includes a peer problems scale, which contains a specific item for peer victimization. If this questionnaire is widely used and registers the presence of peer victimization, it could be used for identifying early warning signs of problematic behaviors indicating that the child is a target of aggressive victimization behaviors, and for studying their risk factors. A warning sign is an observable manifestation that hints of the existence of something that may impact negatively on the person, in this case peer victimization.
Risk factors
To help designing preventive and intervention strategies against peer victimization, it is crucial to determine the risk factors that enable the prediction of the onset of victimization to facilitate an early identification of infants at risk of becoming victims [33]. Several longitudinal studies have identified factors that relate children’s personal characteristics and their home environment with an increased risk of being victimized [34].
The social-ecological model posits that peer victimization involvement is determined by the multiple systems in which youth are embedded [35]. First introduced in the late 1970s [36], this ecological model was initially applied to child abuse [37] and subsequently to youth violence [38]. The model explores the relationship between individual and contextual factors and considers violence as the product of multiple levels of influence on behavior [39]. Understanding how these factors are related to peer victimization is one of the essential steps in the public health approach to preventing this violence [40].
The first level of the ecological model focuses on the intrinsic characteristics of the individual that might increase the likelihood of their either becoming a victim or a perpetrator of violence, and the guiding assumption for these factors is that victimized children behave in ways that invite or reinforce attacks against them [41]. It has been suggested that aggressors “often have a positive outlook on the use of violence to solve problematic situations or get what they want” [42]. This follows the theoretical assumptions underlying proactive aggression. Bullies can be quite adept at identifying victims who will either not retaliate or will be ineffective in their efforts to retaliate, making them more desirable targets [43]. Some children may have temperamental, behavioral and cognitive characteristics that evoke aggressive behavior from others and make them more prone to victimization.
Among children and adolescents, longer exposure to peer victimization has been associated with higher emotional sensitivity, anger, fear and internalizing and unstable affective temperaments [44]. Similarly, psychopathology may contribute to victimization. Over time internalizing (withdrawal, anxiety/depression and hovering peer entry style) and externalizing (aggression, argumentativeness, dishonesty, pushy peer entry style and disruptiveness) behavior problems are associated with victimization [45]. However, prospective studies of young children that examine such antecedent effects are largely lacking [46]. Few studies have tackled the cognitive characteristics of the victimized children. In a Brazilian sample was found that 10- to 11-year-old child peer victims presented difficulties in executive functioning and showed lower cognitive flexibility [47]. Previous studies regarding victimization and ethnic and/or racial differences have produced mixed or inconclusive results [48]. Some studies report that minority youth are more likely to be victimized [49], whereas others have found that the percentage of immigrants and the racial variability of school children do not seem to be risk factors for being victimized [50].
The second level of the ecological model is related to the family environment and explores how proximal social relationships, for example relations with family members, increase the risk of victimization [40]. The impact and relation of family characteristics as risk factors of victimization has gained increased attention in recent years. It has been shown that parental characteristics and adverse lived experiences may have an impact on children, facilitating peer victimization [51]. Several studies have shown that peer victimization is associated with inconsistent, punitive, hostile and/or abusive parenting, high negative expressiveness and high levels of family conflict or violence [52,53]. Also, children growing up in low or mean socioeconomic status (SES) families are at increased risk of victimization [54]. Furthermore, victims from schools in low-SES communities or with a large proportion of students from low-SES families suffer worse long-term mental illness as consequence of the victimization compared to victims from richer families [54]. Additionally, severe prenatal family adversity significantly increased the risk of peer victimization at school [55]. Also, children whose relatives were abused during childhood have an increased risk of being bullied, which may indicate an intergenerational transmission of maltreatment [56]. Last, parental psychopathology also affects the child’s risk of being victimized through complex pathways, such as genetic, neurobiological (e.g., dysregulated stress systems) and social (e.g., modelling) ones [57].
Purpose of the study
The present investigation is based on the premise that children’s behavioral attributes and family factors contribute to their victimization, and has the aim of identifying the children with an increased risk of becoming a victim at the youngest possible age to facilitate timely prevention of the problem. Awareness should be raised about how the instruments at use allow us to identify the problem and the characteristics of the identified children to determine their usefulness in screening for peer victimization [58].
Using a prospective longitudinal design, the present study seeks to contribute new knowledge about the early individual and family risk factors of being peer victimized during the preschool period and seeks to obtain an estimate of the prevalence of these warning signs of peer victimization at 4 to 5 years old, using a widely used screening instrument (SDQ). In alignment with Bronfenbrenner’s ecological model [36], 35 individual and family factors were initially selected to identify which were the most predictive of peer victimization. Selection of the final set of risk factors was made through a predictive modelling approach.
Materials and methods
Study design and procedure
The data was part of a large-scale longitudinal study of behaviour problems in preschool children from age 3, who were screened for behaviour problems and followed up annually until age 9 (the design procedure is detailed in [59]). The two-phase design used involved selecting first a random sample of 2,283 children from the census of in-school 3-year-old preschoolers in Barcelona (Spain) during the 2009–10 academic year (N = 13,578 children). The sample size was determined so that the study was able to detect associations between exposure to different risk factors and the presence of psychopathology equal to or greater than OR = 2.0, accepting an alpha risk of 5% and a statistical power of 80%. Epidemiological studies indicate that the prevalence of psychopathology in children and adolescents (range of ages from 6 to 16 years) of the general population is approximately 15% (p = 0.15) [60]. Likewise, a multiple correlation between covariates of 0.4 (R2 = 0.16) is assumed. Some corrections were applied to the obtained sample: a) correction for homogeneity of children within the same classroom (random clustered sampling) increasing the sample by 20%; b) Correction because double stage design. It was established that screening would classify children into two equally sized groups (50%). Because only 30% of the group of children with negative screening will be followed, it is necessary to proportionally increase the initial number of children to be recruited; and c) Correction for attrition due to refusal of the school to continue participating (assuming 20% of centers will abandon) and due to refusal of subjects, assuming 50% of families will leave the study (as indicated by similar follow-up studies carried out in advanced countries). Exclusion criteria included the child having a generalized developmental disorder, the parents having reading problems or not understanding neither Spanish nor Catalan languages and having planned a move outside Barcelona. All the parents of the initially selected pupils were invited to participate in the study, from which 1,341 families (58.7%) agreed to participate. There were no sex or socioeconomic differences (p = .95 and p = .182) between those who agreed to participate and those who declined, although more participants than refusals attended private schools (p < .001).
The second phase of the sampling consisted in a screening of behavioral problems carried out using the SDQ conduct problems scale [28] (see below) plus four symptoms of DSM-IV oppositional defiant disorder (ODD) (the principal disorder of interest in the original study) not present in the questionnaire (deliberately annoys people, blames others, touchy, angry and resentful). To ensure the presence of children with behavior problems in this second phase, all of those who screened positive for behavioral problems and a random sample of around 30% of children who screened negative (the number of children needed in the negative screening score was calculated to guarantee statistical power) were invited to continue. The final second phase sample included 622 families (10.6% of those invited declined to participate). No differences were found on comparing participants and refusals either by sex (p = .820) or school type (p = .850). The families were interviewed at the school. Interviewers were previously trained and were blind to the children’s screening group. The teachers were asked to complete the questionnaires before the end of the academic year. The Fig 1 shows the sampling procedure.
The research project was approved by the Ethics Commission on Animal and Human Experimentation from the Autonomous University of Barcelona. The parents of the participants children were informed and gave written consent to participate in the study.
Instrumentation
Outcome measures at ages 4 and 5.
Strengths and Difficulties Questionnaire for ages 3–4 (SDQ3-4) [28] for parents and teachers. It is a screening questionnaire for behavioural and emotional problems applied for the purposes of this study when the children were 3 (parents version for control variables), 4 and 5 years old (parents and teachers for the outcome variable). Item 19 - ‘Picked on or bullied by other children’, coded as 0: not true; 1: somewhat true; 2: certainly true—answered by parents and teachers was selected as outcome. We conceptualized that if parents or teachers scored 1 or 2 on this item at both follow-ups at ages 4 and 5 this would be a warning sign for investigating the child’s possible victimization; that is, if the problem as reported by any of the informants was sustained over time, the case would be considered as victimized. Good psychometric properties in the sample have been reported [61], with Cronbach’s alpha = .73 for parents and .82 for teachers at age 3, and slightly better values for ages 4 and 5.
Predictive measures at age 3.
Child Behavior Checklist (CBCL/1½-5) [62]. It is a questionnaire of behavioral and emotional problems reported by parents through 100 items with 3 response options (0: not true, 1: somewhat/sometimes true, 2: very true/often true). Internalizing and externalizing empirical scales, plus the DSM-oriented scales depressive, anxiety, developmental, attention deficit/hyperactivity and oppositional defiant problems, were used for the analyses. Higher scores indicate higher symptomatology.
Behavior Rating Inventory of Executive Function—Preschool Version (BRIEF-P) [63]. It is a standardized questionnaire that assesses behaviors reflecting executive functions in daily life in preschool children. The instrument consists of 63 items on a 3-point scale (1: Never, 2: Sometimes, and 3: Often). Two dimensions which may be associated with social problems, Inhibit (I) and Emotional Control (EM), were used and higher scores indicated greater difficulties. Teachers answered the questionnaire when the children were aged 3. Good psychometric properties have been reported [64].
Children’s Aggression Scale (CAS) [65]. It is a questionnaire that assesses aggressive behavior with 22 items on a 5-point Likert-type scale (0: never to 4: many days). The total score of the teachers’ responses was used as a global index of aggressive behavior. Higher scores indicate greater aggressive behavior.
Children’s Behaviour Questionnaire for ages 3–7 years (CBQ3-7) [66]. It is a questionnaire that measures reactive and self-regulative temperament, with 94 items answered by parents on a 7-point Likert-type scale, ranging from 1 (extremely untrue) to 7 (extremely true). The 3 broad dimensions of temperament: negative affectivity, effortful control and surgency were considered. Higher scores indicate more marked traits. The reported psychometric properties are good [67].
Schedule for Risk Factors (SRF) [68]. It is a computerized structured interview conceived as a compendium of the potential areas of risk of psychopathology that should be evaluated in children. Demographic information including SES [69], pregnancy events, early development and history of abuse suffered by family members were used for this analysis. These risks factors were registered as present/absent as reported by parents. The reported psychometric properties are good [70].
Alabama Parenting Questionnaire for Preschool (APQ-Pr) [71]. It was used to assess parenting practices. It contains 24-item (1: Never to 5: Always) which measures three dimensions of parenting: positive discipline techniques, inconsistent parenting and punitive parenting. Two additional scales extracted from the Evaluation des Pratiques Educatives Parentales–Preschool and Primary School Form (EPEP–PPSF) [72], norms (6 items, labelled as Rules in the EPEP-PPSF) and autonomy (3 items), were added to the instrument with the same response format. In the original EPEP-PPSF, items assessing rules and autonomy emerged as two separated factors and jointly with items assessing positive parenting emerged as a second-order factor of Support; the authors also found low but significant positive correlations between these three measures and desirable traits in the child’s personality. In our sample, an exploratory factor analysis for categorical items conducted with MPlus8.1 showed that the 9-item and 2-factor model with Geomin rotation (χ2 [17] = 108.0, comparative fit index [CFI] = .990, root mean square error of approximation [RMSEA] = .091) for Norms and Autonomy items fitted better than a 9-item and 1-factor model (χ2 [9] = 721.7, CFI = .925, RMSEA = .212); scree test based on eigen values also supported the retention of two factors. All items had salient loading in the expected factor (λ ≥ .65; p ≤ .001). Moreover, in our sample a confirmatory factor analysis conducted with MPlus8.1 for the whole set of items, that is considering a 33-item and 5-factor model, showed acceptable fit (χ2 [485] = 1112.8, CFI = .945, RMSEA = .048), all factor loadings being at least moderate and statistically significant (λ ≥ .34; p ≤ .001); as expected, the higher values for factor correlations were found between norms, autonomy, and positive discipline techniques (r between .35 and .57). For all five measures used in the present study, higher scores indicate higher parental practices in the direction of the label of the scale.
Adult Self-Report (ASR) [73]. It is a questionnaire of dimensional psychopathology for adults aged 18–59. It contains 126 items (0: not true, 1: somewhat or sometimes true, and 2: very true or often true) and was answered by the mothers. DSM-oriented scales were analyzed. Higher scores indicate higher symptomatology.
Second column of Tables 1 and 2 show the Cronbach’s alpha values of the measures in the sample.
Statistical analysis
The software used for statistical analysis was Stata 15 for Windows. To reflect population prevalence (as final participant selection was conditioned by the presence/absence of behavior problems), the analyses were weighted by the inverse probability of being selected in the second phase of the sampling design.
A two-stage modelling procedure [74] was used to determine the best predictive model of 4- to 5-year-old warning signs of peer victimization from the 35 predictors pre-selected at age 3 according to the literature. First, several independent binary logistic regression models were estimated, grouping the predictors by content (demographics, individual and family risk factors). Because of the large number of predictors included in this first stage, the false discovery rate (FDR) [75] was applied to control the Type I error and to know which predictors remained significant after correction. Second, predictors with an FDR-corrected significance p-value below .10 were selected and a backward stepwise binary logistic regression model was conducted. Predictors from this model with p-values below .05 were used to calculate the area under the ROC curve (AUC), which indicated the proportion of correct predictions of warning signs of victimization achieved with the selected predictors. AUC values above .75 are usually considered as indicating a good prediction capability. This final model included as covariates parents’ and teachers’ answers to item 19 of the SDQ (“Picked on or bullied by other children”) at age 3 to adjust the odds-ratios of victimization at baseline. The predictive capability of the model was calculated through Pseudo-R2 [76] and translated in terms of ordinary R2 using the same author’s approach.
Since no cross-validation weighted commands have been developed for Stata, the final predictive model was validated using a repeated random sub-sampling cross-validation procedure [77], which produces a summary measure of loss of prediction.
The Hosmer-Lemeshow goodness-of-fit test was verified for each logistic model and the absence of outliers and influential observations was also confirmed [78].
Results
A total of 602 children were available at age 4 (96.8% from the 622 children starting the study at age 3) and 577 (92.8%) remained at age 5. Participants and drop-outs at age 5 were statistically equal in sex (p = .297), but participants had a higher socioeconomic status (SES) than drop-outs (p = .002). Parents’ information was obtained from mothers (67.6%), fathers (7.6%) or both (24.8%), with a mean age of 37.6 years. Only 5 out of 577 teachers were male. Table 3 shows the demographic characteristics of participants for the total sample and separately by absence/presence of victimization warning signs.
Prevalence of warning signs of peer victimization
According to the answers given to SDQ-item 19 (“Picked on or bullied by other children”) by the 577 participants remaining in the study at age 5, and applying our definition of warning signs of peer victimization (if answers were 1 or 2), the parents reported warning signs at one follow-up (4 or 5 years old) for 15.9% of preschoolers and at both follow-ups for 3.0% of preschoolers. Teachers reported comparable results, with 16.4% of children presenting warning signs at only one age and 1.4% at 4 and 5 years old. To minimize lack of sensitivity, a child was considered to display warning signs of peer victimization if parents or teachers reported the item positively at both ages 4 and 5. On applying this definition, 24 children (4.16%, 95% CI [2.68, 6.13]) were classified as displaying warning signs of peer victimization. Chi-square tests revealed that there were no differences by sex or ethnicity in the prevalence of warning signs of peer victimization, but the risk increased as SES decreased (Table 3).
Initial selection of risk factors of warning signs of peer victimization at 4 to 5 years old
Table 1 presents descriptive statistics for the individual characteristic predictors (demographics, dimensional psychopathology, executive functioning, aggressive behavior and temperament) for children with and without warning signs of peer victimization separately; the last three columns show the results of the first stage of the modelling procedure, consisting of the estimation of several independent logistic regression models to predict warning signs of peer victimization at 4 to 5 years old (one model for each set of predictors: demographics, dimensional psychopathology, etc.). Four individual attributes showed significant odds-ratios: low or medium-low SES, CBCL-internalizing, CBCL-developmental problems and BRIEF-P emotional control difficulties. After applying the FDR correction to control the increment of the type I error due to the high number of statistical tests performed, two predictors remained significant at .10 value: SES and BRIEF-P emotional control.
Similarly, Table 2 shows descriptive statistics for the family and early predictors (pregnancy, early development, history of abuse, parenting practices and maternal psychopathology). The last three columns show the odds ratios obtained from the first phase of the modelling strategy, estimating independent logistic regression models to predict warning signs of victimization at 4 to 5 years old from each set of predictors (pregnancy, early development, etc.). Five significant odds ratios were found: economic problems during pregnancy, child’s early difficulties in making friends, a history of childhood abuse among relatives, lower norms and higher ASR-somatic problems as reported by the mother. Statistical significance remained below .10 after FDR correction for three predictors: difficulties in making friends, childhood abuse among relatives and lower norms.
Final predictive model with risk factors of warning signs of peer victimization at ages 4 to 5
The five predictors with FDR-corrected p-values under .10 (two individual and three family predictors) were selected as the maximum logistic regression model in the second stage of the modelling procedure. To select the significant predictors from these five a backward stepwise regression was applied, which retained four significant predictors (Table 4). This final model was adjusted by the baseline presence/absence of item 19 reported by both parents and teachers. The resulting odds ratios show that the risk of presence of warning signs of peer victimization at 4 to 5 years old increased for children with a low or medium-low SES (p = .001), higher scores in BRIEF-P emotional control (p = .004), difficulties in making friends (p = .015) and parenting practices characterized by lower scores in APQ norms (p < .001).
This final multiple model, which included four terms plus the two adjusting terms, obtained a Pseudo-R2 = .22, which according to [76] is approximately equivalent to an R2 = .50 of a linear regression. Fig 2 shows the corresponding ROC curve, which reflects the accuracy of these four variables to predict the presence of warning signs of victimization. The AUC = .78 (95% CI [.75, .81]) is statistically better than a classification made by chance (p < .001) and could be considered as almost excellent [74]. Generalizability of the predictive model could be considered acceptable as loss of prediction obtained through repeated random sub-sampling cross-validation was only .044 in terms of AUC.
Discussion
The purpose of this study was to report on the prevalence and risk factors of warning signs of preschool victimization as assessed by the SDQ, a widely used screening questionnaire reported by parents and teachers. According to the endorsement of item 19 (“Picked on or bullied by other children”), 4.2% of 4- to 5-year-old in-school preschoolers suffered persistent victimization. Although few studies have reported the prevalence of warning signs of victimization in preschoolers, our estimation of prevalence (4.2%) is in line with that reported by previous studies [10,27,79] for children of similar ages in other cultures using one item of a questionnaire for assessing the rates of bullying. Our assessment of victimization was also done with one item of a widely used instrument, the SDQ.
Given that aggressive behavior tolerance at any age should be near 0, our results indicating a prevalence of warning signs of victimization in preschoolers equal to 4.2% showed that from preschool age there was a considerable number of children persistently exposed to being picked on or bullied by other children. The adverse consequences of preschool victimization are manifested in children and adolescents in a deteriorated mental health (depression, anxiety, suicidal ideation and suicide attempts) and in the increase of psychosocial problems [80].
A variety of individual and family factors at age 3 increased the risk of parents and teachers reporting persistent victimization: pertaining to a low socioeconomic status family, problems in emotional control, early difficulties in making friends and parental practices characterized by lower norms. These variables explained almost 50% of the variance of parents and teachers reporting victimization with a near to excellent ability to discriminate between the groups with and without warning signs of potential victimization.
Low SES at age 3 was more frequent in the at-risk group. Several studies showed that being involved in peer victimization either as victim or perpetrator is more frequent in children and adolescents with a lower SES [10,51]. Families with a low SES might have influenced children’s involvement in victimization in several ways. For instance, they tend to have a lower participation in social activities [81]. In low SES families, parents tend to have less involvement in their children’s education, practicing low levels of academic socialization with them [82]. The educational level of parents was included in the family SES calculation because education not only influences economic status, but also reflects non-economic social characteristics, such as intelligence, knowledge, social norms and values, literacy and competence in problem solving [10], all of which are aspects that could be related to child raising behavior and, consequently, to children’s limitations in the development of social skills and coping strategies [10,51]. Families living with this adversity would need victimization surveillance.
Difficulties in emotional control reported by teachers was found to increase the risk of parents and teachers reporting that the child was being picked on or bullied. In other words, children who overreact to small problems, are explosive, have outbursts for little reason, are angry or tearful or react very strongly to situations, are targets for victimization. This result is consistent with previous studies, among them a methodologically similar study with parents and teachers as informant and children from 3 to 5 years showing a high association between emotion dysregulation and higher rates of victimization [83]. This finding agrees with predictions derived from the social process model [84] because emotional dysregulation may cause peers isolation, negative judgments and consequently increase the risk for victimization. This path from emotional control to victimization through social processes was also evidenced in clinical samples of children with a diagnose of attention-deficit/hyperactivity disorder [85] and children with early manifestations of oppositional defiant disorder [46]. Both disorders are characterized by high emotional reactivity, demonstrating that these disorders at preschool age predicted children’s risk of bullying involvement in the first years of elementary school. Children who show high emotional reactivity at preschool entry may therefore require special vigilance at school.
Difficulty in making friends was another strong predictor of parents and teachers reporting that the child was victimized. From early childhood, social competence is an important protective factor for children’s adjustment, helping them to engage in shared activities, adapt to different situations, develop friendships, respect turns, regulate emotionality, collaborate, show prosocial behavior, acquire a good reputation among peers and be accepted by them [86]. Social competence is thus a solid base for building good relationships and when this is not well developed the child is at risk of being isolated and isolated children are also a target for victimization [87].
Regarding parenting practices, our results are in line with previous studies supporting that poor parental supervision, harsh parental discipline, low parental involvement with the child and marital discord are associated with childhood aggression and bullying [88]. Relatively little is known, however, about how this influence operates [89]. A low parental educational level in social norms may interfere with proper socialization as the child has a deficit in references about how to behave with others and this has also been related to victimization in general population [90].
Limitations
This study has some limitations that must be highlighted. We did not include information reported from the child because the young age of the participants and the methods used hampered their participation. In older children, parents are usually unaware of peer victimization involvement, in part because young people do not always divulge their experiences with victimization to their parents [91], whereas parental follow-up is tighter with preschoolers. Having no information directly from the child, however, could mean that some cases of victimization did not come to light.
A second limitation is that a high percentage of the participant attrition in the follow-up at age 5 was among low socioeconomic status families. This could have resulted in a slight under-estimation of the prevalence of victimization.
Another limitation is in reference to the social-ecological model, because we have not taken into account the community contexts in which social relationships are embedded. The classroom environment had a great influence on the amount of victimization reported by peers, indicating that there is something in the class context that potentiates or inhibits bullying [92].
On the other hand, this study also has several strengths. Peer victimization was assessed using multiple informants (parents and teachers) and considers risk factors from multiple domains (individual and family). Last, much of the previous work on victims has been conducted with older children and there are only a limited number of studies that focus on the preschool period.
Conclusions
Peer-victimization starts at preschool ages, so early intervention in the preschool context is crucial if we are to break the cycle of victimization in later childhood [34]. Since both individual and family risk factors of different nature are present in the victims, to stop the victimization from escalating requires a broad and multidisciplinary approach.
Implications for school health
School is a context where peer victimization may occur, and this markedly alters community mental health. The development of programs designed to prevent its escalation, reduce its negative impact on children’s development and promote adjustment are needed. Early identification of both the problem itself, using screening questionnaires for detecting warning signs, and of the problems associated with victimization dynamics may enable closer monitoring of the risk groups as a preventative measure. The prevention of victimization and its consequences can be improved by focusing on risk groups at school in early life. Our results suggest that victimization intervention programs should consider the role of the victims’ personal characteristics in increasing vulnerability to victimization and the importance of including families in school-based intervention programs aimed at reducing the difficulties experienced by victimized children. In summary, research identifying the risk factors for victimization helps to target interventions to deal with the key factors contributing to the problem, thus maximizing their effectiveness [92].
References
- 1. Cooley JL, Fite PJ, Pederson CA. Bidirectional associations between peer victimization and functions of aggression in middle childhood: Further evaluation across informants and academic years. J Abnorm Child Psychol. 2017; pmid:28217827
- 2. Salmivalli C. Bullying and the peer group: A review. Aggress Violent Behav. 2010;15: 112–120.
- 3. Copeland WE, Wolke D, Angold A, Costello EJ. Adult Psychiatric Outcomes of Bullying and Being Bullied by Peers in Childhood and Adolescence. JAMA Psychiatry. 2013;70: 419. pmid:23426798
- 4. Takizawa R, Maughan B, Arseneault L. Adult health outcomes of childhood bullying victimization: evidence from a five-decade longitudinal British birth cohort. Am J Psychiatry. 2014;171: 777–84. pmid:24743774
- 5. Kochenderfer BJ, Ladd GW. Peer victimization: cause or consequence of school maladjustment? Child Dev. 1996;67: 1305–17. http://www.ncbi.nlm.nih.gov/pubmed/8890485 pmid:8890485
- 6. Bowes L, Maughan B, Caspi A, Moffitt TE, Arseneault L. Families promote emotional and behavioural resilience to bullying: evidence of an environmental effect. J Child Psychol Psychiatry. 2010;51: 809–817. pmid:20132419
- 7. Barker ED, Arsenault L, Brendgen M, Fontaine N, Maughan B. Joint Development of Bullying and Victimization in Adolescence. J Am Acad Child Adolesc Psychiatry. 2008;47: 1030–1038. pmid:18665001
- 8. Juvonen J, Graham S. Bullying in schools: the power of bullies and the plight of victims. Annu Rev Psychol. 2014;65: 159–85. pmid:23937767
- 9. Herba CM, Ferdinand RF, Stijnen T, Veenstra R, Oldehinkel AJ, Ormel J, et al. Victimization and suicide ideation in the TRAILS study: specific vulnerabilities of victims. J Child Psychol Psychiatry. 2008;49: 867–876. pmid:18492041
- 10. Jansen PW, Verlinden M, Berkel AD-V, Mieloo C, Van Der Ende J, Veenstra R, et al. Prevalence of bullying and victimization among children in early elementary school: Do family and school neighbourhood socioeconomic status matter? BMC Public Health. 2012;12: 494. pmid:22747880
- 11. Crick NR, Bigbee MA. Relational and overt forms of peer victimization: A multiinformant approach. J Consult Clin Psychol. 1998;66: 337–347. pmid:9583337
- 12. Nansel TR, Overpeck M, Pilla RS, Ruan WJ, Simons-Morton B, Scheidt P. Bullying behaviors among US youth—Prevalence and association with psychosocial adjustment. Jama-Journal Am Med Assoc. 2001;285: 2094–2100. pmid:11311098
- 13. Kaltiala-Heino R, Rimpelä M, Rantanen P, Rimpelä A. Bullying at school-an indicator of adolescents at risk for mental disorders. J Adolesc. 2000;23: 661–674. pmid:11161331
- 14. Arseneault L, Walsh E, Trzesniewski K, Newcombe R, Caspi A, Moffitt TE. Bullying victimization uniquely contributes to adjustment problems in young children: a nationally representative cohort study. Pediatrics. 2006;118: 130–138. pmid:16818558
- 15. Strøm IF, Aakvaag HF, Birkeland MS, Felix E, Thoresen S. The mediating role of shame in the relationship between childhood bullying victimization and adult psychosocial adjustment. Eur J Psychotraumatol. 2018;9: 1418570. pmid:29372013
- 16. Laditka JN, Laditka SB. Adverse Childhood Circumstances and Functional Status Throughout Adult Life. J Aging Health. 2018;30: 1347–1368. pmid:28643582
- 17. Sigurdson JF, Wallander J, Sund AM. Is involvement in school bullying associated with general health and psychosocial adjustment outcomes in adulthood? Child Abuse Negl. 2014;38: 1607–17. pmid:24972719
- 18. Vlachou M, Andreou E, Botsoglou K, Didaskalou E. Bully/victim problems among preschool children: A review of current research evidence. Educ Psychol Rev. 2011;23: 329–358.
- 19. Hanish LD, Ryan P, Martin CL, Fabes RA. The social context of young children’s peer victimization. Soc Dev. 2005;14: 2–19. pmid:16493454
- 20. Beauchesne MA, Farquharson PE, Beauchemin P. Buddies Not Bullies: an approach to bullying prevention education in preschool children, their families and the community. Nurs Child Young People. 2017;29: 41–45. pmid:28395634
- 21. Huang H, Liu Y, Chen Y. Preservice Preschool Teachers’ Responses to Bullying Scenarios: The Roles of Years of Study and Empathy. Front Psychol. 2018;9: 175. pmid:29515492
- 22. Björkqvist K. Different Names, Same Issue. Soc Dev. John Wiley & Sons, Ltd (10.1111); 2001;10: 272–274.
- 23. Monks CP, Smith PK, Swettenham J. Psychological correlates of peer victimisation in preschool: social cognitive skills, executive function and attachment profiles. Aggress Behav. John Wiley & Sons, Ltd; 2005;31: 571–588.
- 24. Rigby K, Smith PK. Is school bullying really on the rise? Soc Psychol Educ An Int J. 2011;14: 441–455.
- 25. Craig W, Harel-Fisch Y, Fogel-Grinvald H, Dostaler S, Hetland J, Simons-Morton B, et al. A cross-national profile of bullying and victimization among adolescents in 40 countries. Int J Public Health. 2009;54. pmid:19623475
- 26. Monks CP, Smith PK, Swettenham J. Aggressors, victims, and defenders in preschool: Peer, self-, and teacher reports. Merrill Palmer Q. 2003;49: 453–469.
- 27. Ilola A-M, Lempinen L, Huttunen J, Ristkari T, Sourander A. Bullying and victimisation are common in four-year-old children and are associated with somatic symptoms and conduct and peer problems. Acta Paediatr Int J Paediatr. 2016;105: 522–528. pmid:26741067
- 28. Goodman R. The Strengths and Difficulties Questionnaire: A Research Note. J Child Psychol Psychiatry. 1997;38: 581–586. pmid:9255702
- 29. Dahlberg A, Ghaderi A, Sarkadi A, Salari R. SDQ in the Hands of Fathers and Preschool Teachers—Psychometric Properties in a Non-clinical Sample of 3–5-Year-Olds. Child Psychiatry Hum Dev. Springer US; 2019;50: 132–141. pmid:29959588
- 30. Fält E, Sarkadi A, Fabian H. Exploring Nurses’, Preschool Teachers’ and Parents’ Perspectives on Information Sharing Using SDQ in a Swedish Setting—A Qualitative Study Using Grounded Theory. PLoS One. Public Library of Science; 2017;12: e0168388. pmid:28076401
- 31. De Los Reyes A, Henry DB, Tolan PH, Wakschlag LS. Linking informant discrepancies to observed variations in young children’s disruptive behavior. J Abnorm Child Psychol. NIH Public Access; 2009;37: 637–52. pmid:19247829
- 32. De Los Reyes A, Kazdin AE. Informant Discrepancies in the Assessment of Childhood Psychopathology: A Critical Review, Theoretical Framework, and Recommendations for Further Study. Psychol Bull. 2005;131: 483–509. pmid:16060799
- 33. Álvarez-García D, García T, Núñez JC. Predictors of school bullying perpetration in adolescence: A systematic review. Aggress Violent Behav. Pergamon; 2015;23: 126–136.
- 34. Bowes L, Maughan B, Ball H, Shakoor S, Ouellet-Morin I, Caspi A, et al. Chronic bullying victimization across school transitions: the role of genetic and environmental influences. Dev Psychopathol. 2013;25: 333–346. pmid:23627948
- 35. Espelage D, Swearer S. Research on School Bullying and Victimization: What Have We Learned and Where Do We Go from Here? School Psych Rev. 2003;32: 365–383.
- 36.
Bronfenbrenner U. The Ecology of Human Development—Experiments by Nature and Design [Internet]. Harvard University Press; 1979. https://doi.org/10.1080/00131728109336000
- 37. Garbarino J, Crouter A. Defining the Community Context for Parent-Child Relations: The Correlates of Child Maltreatment. Child Dev. WileySociety for Research in Child Development; 1978;49: 604–616.
- 38.
Garbarino J. Adolescent development : an ecological perspective [Internet]. C.E. Merrill Pub. Co; 1985. https://books.google.es/books/about/Adolescent_development.html?id=6JQEAQAAIAAJ&redir_esc=y
- 39. Dahlberg LL, Krug EG. Violence a global public health problem. Cien Saude Colet. ABRASCO—Associação Brasileira de Saúde Coletiva; 2006;11: 277–292.
- 40.
Dahlberg L, Krug E. Violence: a global public health problem. In: Krug E, Dahlberg L, Mercy J, Zwi A, Lozano R, editors. World Report on Violence and Health. John Wiley & Sons, Ltd; 2002. https://doi.org/10.1002/casp.740
- 41. Hodges E, Malone M, Perry D. Individual risk and social risk as interacting determinants of victimization in the peer group. Dev Psychol. 1997;33: 1032–1039. pmid:9383625
- 42. Carney AG, Merrell KW. Bullying in Schools. Perspectives on understanding and preventing an international problem. Sch Psychol Int. 2001;22: 364–382.
- 43. Griffin RS, Gross AM. Childhood bullying: Current empirical findings and future directions for research. Aggress Violent Behav. 2004;9: 379–400.
- 44. Frizzo MN, Bisol LE, Lara DR. Bullying victimization is associated with dysfunctional emotional traits and affective temperaments. J Affect Disord. 2013;148: 48–52. pmid:23245468
- 45. Hodges E, Perry D. Personal and interpersonal antecedents and consequences of victimization by peers. J Pers Soc Psychol. 1999;76: 677–85. http://www.ncbi.nlm.nih.gov/pubmed/10234851 pmid:10234851
- 46. Verlinden M, Jansen PW, Veenstra R, Jaddoe VW V, Hofman A, Verhulst FC, et al. Preschool Attention-Deficit/Hyperactivity and Oppositional Defiant Problems as Antecedents of School Bullying. J Am Acad Child Adolesc Psychiatry. Elsevier Inc.; 2015;54: 571–579. pmid:26088662
- 47. Medeiros W, Torro-Alves N, Malloy-Diniz LF, Minervino CM. Executive Functions in Children Who Experience Bullying Situations. Front Psychol. 2016;7. pmid:27616998
- 48. Bradshaw CP, Sawyer AL, O’Brennan LM. A Social Disorganization Perspective on Bullying-Related Attitudes and Behaviors: The Influence of School Context. Am J Community Psychol. 2009;43: 204–220. pmid:19333749
- 49. Sawyer AL, Bradshaw CP, O’Brennan LM. Examining Ethnic, Gender, and Developmental Differences in the Way Children Report Being a Victim of “Bullying” on Self-Report Measures. J Adolesc Heal. 2008;43: 106–114. pmid:18639783
- 50. Wolke D. Bullying involvement in primary school and common health problems. Arch Dis Child. 2001;85: 197–201. pmid:11517098
- 51. Perren S, Stadelmann S, von Klitzing K. Child and family characteristics as risk factors for peer victimization in kindergarten. Schweizerische Zeitschrift fur Bild. 2009;31: 13–32.
- 52. Burk LR, Park J, Armstrong JM, Klein MH, Goldsmith HH, Zahn-Waxler C, et al. Identification of early child and family risk factors for aggressive victim status in first grade. J Abnorm Child Psychol. 2008;36: 513–526. pmid:18092191
- 53. Mohr A. Family variables associated with peer victimization: Does family violence enhance the probability of being victimized by peers? Swiss J Psychol. 2006;65: 107–116.
- 54. Due P, Damsgaard MT, Lund R, Holstein BE. Is bullying equally harmful for rich and poor children?: a study of bullying and depression from age 15 to 27. Eur J Public Health. 2009;19: 464–469. pmid:19587227
- 55. Lereya ST, Wolke D. Prenatal family adversity and maternal mental health and vulnerability to peer victimisation at school. J Child Psychol Psychiatry. 2013;54: 644–652. pmid:23121554
- 56. Pears KC, Capaldi DM. Intergenerational transmission of abuse: a two-generational prospective study of an at-risk sample. Child Abuse Negl. 2001;25: 1439–1461. pmid:11766010
- 57. Goodman A, Joyce R, Smith JP. The long shadow cast by childhood physical and mental problems on adult life. Proc Natl Acad Sci U S A. 2011;108: 6032–6037. pmid:21444801
- 58. Vessey J, Strout TD, DiFazio RL, Walker A. Measuring the youth bullying experience: A systematic review of the psychometric properties of available instruments. J Sch Health. 2014;84: 819–843. pmid:25388599
- 59. Ezpeleta L, de la Osa N, Doménech JM. Prevalence of DSM-IV disorders, comorbidity and impairment in 3-year-old Spanish preschoolers. Soc Psychiatry Psychiatr Epidemiol. 2014;49: 145–155. pmid:23595297
- 60. Merikangas KR, He J-P, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication—Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry. NIH Public Access; 2010;49: 980–9. pmid:20855043
- 61. Ezpeleta L, Granero R, de la Osa N, Penelo E, Doménech JM. Psychometric properties of the Strengths and Difficulties Questionnaire3-4 in 3 year-old preschoolers. Compr Psychiatry. 2013;54: 282–291. pmid:22999518
- 62.
Achenbach TM, Rescorla LA. Manual for the ASEBA preschool-age forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth & Families; 2000.
- 63.
Gioia GA, Espy KA, Isquith PK. Behavior Rating Inventory of Executive Function—Preschool version. Lutz, FL: Psychological Assessment Resources; 2003.
- 64. Ezpeleta L, Granero R, Penelo E, de la Osa N, Domènech JM. Behavior Rating Inventory of Executive Functioning–Preschool (BRIEF-P) Applied to Teachers. J Atten Disord. 2015;19: 476–488. pmid:23264366
- 65.
Halperin JM, McKay KE. Children’s Aggression Scale. Lutz, FL: Psychological Assessment Resources; 2008.
- 66. Putnam SP, Rothbart MK. Development of short and very short forms of the Children’s Behavior Questionnaire. J Pers Assess. 2006;87: 103–113.
- 67. de la Osa N, Granero R, Penelo E, Domènech JM, Ezpeleta L. The Short and Very Short Forms of the Children’s Behavior Questionnaire in a Community Sample of Preschoolers. Assessment. 2014;21: 463–476. pmid:24235176
- 68.
Unitat d’Epidemiologia i de Diagnòstic en Psicopatologia del Desenvolupament. Protocolo de Factores de Riesgo. Versión para preescolares. Barcelona: Universitat Autònoma de Barcelona, Departament de Psicologia Clínica i de la Salut; 2009.
- 69.
Hollingshead AB. Four factor index of social status [Internet]. New Haven, CT: Unpublished manuscript, Yale University, Department of Sociology; 1975. http://www.yale-university.com/sociology/faculty/docs/hollingshead_socStat4factor.pdf
- 70.
Guillamón N. Fiabilidad entre entrevistadores del Protocolo de Factores de Riesgo-Versión niños. [Inter-rater fiability of the risk factors protocol-Children version]. 1999.
- 71. de la Osa N, Granero R, Penelo E, Doménech JM, Ezpeleta L. Psychometric properties of the Alabama Parenting Questionnaire–Preschool revision (APQ-Pr) in 3 year-old Spanish preschoolers. J Child Fam Stud. 2014;23: 776–784.
- 72. Meunier J-C, Roskam I. Validation of the preschool and primary school form of a questionnaire assessing parents’ childrearing behavior. J Clin Child Adolesc Psychol. 2009;38: 166–75. pmid:19130365
- 73.
Achenbach TM, Rescorla LA. Manual for the ASEBA adult forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth & Families; 2003.
- 74.
Hosmer DW, Lemeshow S, Sturdivant RX. Applied logistic regression. 3rd Ed. New Jersey: John Wiley & Sons; 2013.
- 75. Benjamini Y, Hochberg Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. J R Stat Soc Ser B. 1995;7: 289–300.
- 76.
McFadden D. Conditional logit analysis of qualitative choice behavior. In: Zarembka P, editor. Frontiers in Econometrics. New York: Academic Press.; 1973. pp. 105–142.
- 77.
Dubitzky W, Granzow M, Berrar D. Fundamentals of data mining in genomics and proteomics. New York: Springer; 2007.
- 78.
Kleinbaum DG, Klein M. Logistic Regression. A self-learning text. 3rd ed. New York: Springer-Verlag; 2010.
- 79. Øksendal E, Brandlistuen RE, Holte A, Wang MV. Peer-Victimization of Young Children With Developmental and Behavioral Difficulties—A Population-Based Study. J Pediatr Psychol. 2019;44: 589–600. pmid:30816959
- 80. Moore SE, Norman RE, Suetani S, Thomas HJ, Sly PD, Scott JG. Consequences of bullying victimization in childhood and adolescence: A systematic review and meta-analysis. World J psychiatry. Baishideng Publishing Group Inc; 2017;7: 60–76. pmid:28401049
- 81. Stevens V, De Bourdeaudhuij I, Van Oost P. Relationship of the Family Environment to Children’s Involvement in Bully/Victim Problems at School. J Youth Adolesc. Kluwer Academic Publishers-Plenum Publishers; 2002;31: 419–428.
- 82. Carolan B V., Wasserman SJ. Does Parenting Style Matter? Concerted Cultivation, Educational Expectations, and the Transmission of Educational Advantage. Sociol Perspect. SAGE PublicationsSage CA: Los Angeles, CA; 2015;58: 168–186.
- 83. Godleski SA, Kamper KE, Ostrov JM, Hart EJ, Blakely-McClure SJ. Peer Victimization and Peer Rejection During Early Childhood. J Clin Child Adolesc Psychol. 2015;44: 380–392. pmid:25133659
- 84.
Boivin M, Hymel S, Hodges EVE. Toward a process view of peer rejection and harassment. In: Juvonen J, Graham S, editors. Peer harassment in school: The plight of the vulnerable and victimized. New York, NY: US: The Guilford Press; 2001. pp. 265–289.
- 85. Hoza B, Mrug S, Gerdes AC, Hinshaw SP, Bukowski WM, Gold JA, et al. What Aspects of Peer Relationships Are Impaired in Children With Attention-Deficit/Hyperactivity Disorder? J Consult Clin Psychol. 2005;73: 411–423. pmid:15982139
- 86. Denham SA, Blair KA, Demulder E, Levitas J, Sawyer K, Auerbach-Major S, et al. Preschool Emotional Competence: Pathway to Social Competence? Child Dev. 2003;74: 238–256. pmid:12625448
- 87. Perren S, Alsaker FD. Social behavior and peer relationships of victims, bully-victims, and bullies in kindergarten. J Child Psychol Psychiatry. 2006;47: 45–57. pmid:16405640
- 88. Bibou-Nakou I, Tsiantis J, Assimopoulos H, Chatzilambou P. Bullying/victimization from a family perspective: a qualitative study of secondary school students’ views. Eur J Psychol Educ. 2013;28: 53–71.
- 89. Jiménez TI, Musitu G, Ramos MJ, Murgui S. Community involvement and victimization at school: An analysis through family, personal and social adjustment. J Community Psychol. 2009;37: 959–974.
- 90. Baldry AC, Farrington DP. Protective factors as moderators of risk factors in adolescence bullying. Soc Psychol Educ. 2005;8: 263–284.
- 91. Houndoumadi A, Pateraki L. Bullying and bullies in Greek elementary schools: Pupils’ attitudes and teachers’/parents’ awareness. Educ Rev. 2001;53: 19–26.
- 92. Saarento S, Kärnä A, Hodges E, Salmivalli C. Student-, classroom-, and school-level risk factors for victimization. J Sch Psychol. 2013;51: 421–434. pmid:23816233