Selection criteria.

S3 Appendix contains the full details of the selection criteria and S4 Appendix provides the reasons for exclusion of certain citations.

Training the appraisers.

Three trained individuals (CS, SR, KTA) formed two pairs of appraisers and received the same training with the AGREE II online tutorial [22]. Prior to evaluating the JIA CPGs, the appraisers reviewed a CPG on osteoarthritis using the AGREE II instrument to familiarize themselves with the instrument. Each appraiser independently scored the CPGs using the AGREE II items and then assigned an overall quality rating to the CPG using the same 7-point scale. For each CPG, appraisers also indicated whether they: recommended its use by clinicians for practice, recommended its use with some modifications, or did not recommend its use.

Grading of the items and domains.

The AGREE II user manual was used by appraisers to refer to the criteria for each item [21]. A grade was assigned for each item using a 7-point scale indicating the level of agreement with each statement about the methodological quality of the CPGs. A score of 1 was given if none of the criteria for an item were met or the item was reported very poorly, while a score of 7 was given for an item if it met all the criteria and was well reported. The domain score was calculated using the sum of the item scores in a given domain and transforming the number into a percentage of the maximum score that domain could obtain. For this review, a high quality CPG required a score of 60% for rigor of development (domain 3) as well as 60% in any two other domains [23–25].

Calculating the domain scores.

After all appraisers finished grading the CPGs, their scores were entered into an AGREE II score calculator developed by the Capacity Enhancement Program at McMaster University [26]. An overall methodological quality score of all items in all domains was computed on a 7-point scale for each CPG. The concordance calculator was used to ensure the correct calculation of domain scores ( (SD)) from the equation provided in the AGREE II user’s manual [21].

Discrepancies in rater scores.

If the concordance calculator computed a medium or high discrepancy in domain scoring among the three appraisers, a fourth appraiser (LB) resolved the discrepancies. A medium discrepancy was defined as individual appraisers’ scores for a domain being 1.5 to 2 standard deviations from the mean domain score. A high discrepancy was greater than 2 standard deviations from the mean domain score [26]. To reduce the level of error, the three appraisers independently reviewed for a second time all items where there were medium discrepancies or where scores differed by three or more points between at least two appraisers.

Inter-rater reliability.

Inter-rater reliability was examined by comparing the individual item scores of each appraiser and ensuring there was only a low discrepancy (less than 1.5 standard deviations from the mean domain score) with the concordance calculator [26].

Summary of CPG recommendations.

The recommendations provided by each CPG were presented based on types of patients (e.g. clinical scenarios or JIA onset types), rather than by intervention to take into consideration the complexity of care for patients with JIA. Any treatment algorithms presented by the CPGs were also summarized to indicate the progression of treatment for JIA that the CPGs recommended.

Domain 1: Scope and purpose.

The CPGs had scores above 60% for scope and purpose except for GKJR [27] which had a score of 57%. The main reason for the score below 60% was that there was no detailed search strategy provided and this made the health questions covered by the CPG unclear. The overall objectives of the CPG and the population to whom the CPG applied were in general well described for all CPGs.

Domain 2: Stakeholder involvement.

All three CPGs scored above 60%, although the mean quality score was lower than domain 1. This is largely due to unclear descriptions of how the CPGs sought the views and preferences of the target population. A description of the CPG development group was acceptable, though there was not always a description of disciplines or individual roles of those involved in the included CPGs.

Domain 3: Rigor of development.

ACR [29,30] and RACGP [28] were very similar in their quality scores, which were both above 60%. GKJR [27] received a lower score of 56%. The main reason for the borderline score was there was no procedure given for updating the CPG, which led to scores of 1 and 2. The other two CPGs still had moderate scores for item 14 because the information provided was somewhat limited. In general, the CPGs used systematic methods in searching for evidence, as well as provided an obvious link between recommendations and their evidence.

Domain 4: Clarity of presentation.

This domain received high scores from all three CPGs and the highest mean quality score (SD) was 87.00% (9.64%) (Table 1). Each CPG provided a type of treatment algorithm for pharmacological interventions and provided consensus statements for which interventions were recommended.

Domain 5: Applicability.

Consistent across the CPGs, this domain had the lowest scores ranging from 8% to 19% (Table 1). All of the CPGs scored very poorly because they did not clearly state the costs and resources that were needed in order to implement the CPG, which was required by the AGREE II instrument [21,27–29]. Criteria for monitoring the implementation of recommendations and assessing the CPG’s impact were either not provided or simply involved referring readers to government documents [27].

Domain 6: Editorial independence.

Of all the domains, editorial independence had the second lowest mean quality score at 43.33% (7.02%) (Table 1). While the funding body may have been mentioned, ACR and GKJR did not provide an explicit statement to declare that the funding body did not have an influence on the guideline content [27,29] and it was not entirely clear to the appraisers whether one of the members in the RACGP working group, who was an advisor for the funding source, influenced the CPG [28]. There was frequently missing information for recording the competing interests for members of the guideline development group in all CPGs.

Inter-rater reliability.

The domain scores entered into the concordance calculator showed low discrepancy between pairs of assessors except for stakeholder involvement (domain 2), which required review for the ACR [29,30] and the GKJR [27] CPGs. The appraisers were asked to review their scores in domain 2 and determine if they had overlooked or misunderstood information provided in the CPGs when scoring the items. No change in the appraisers’ scores was made if they had originally found the relevant information and had understood it. Upon making these revisions, the medium discrepancy level (individual scores being between 1.5 and 2 standard deviations from the mean) for these two domains were resolved. In addition, each appraiser’s raw data scores for all 23 items were fairly consistent with only a few instances when two appraisers had scores varying by three points on the 7-point scale.

RACGP 2009.

The RACGP CPG was focused on providing recommendations within an Australian healthcare context for general practitioners (GPs) and other multidisciplinary healthcare professionals who provide support and/or care for patients with JIA. The population of interest consisted of patients less than 16 years who have at least one joint that is painful and swollen [28]. The CPG directs GPs on early diagnosis of JIA, initial treatment options, and timing of referral to a pediatric rheumatologist or other healthcare professionals, all with a multidisciplinary focus [28]. Pharmacological interventions considered were oral NSAIDs, topical NSAIDs, simple analgesics (e.g. acetaminophen), weak opioids, and complementary and alternative medicines. DMARDs and biologics were not considered as they are beyond the scope of interventions prescribed by GPs. Thus, GPs were directed to refer patients to a pediatric rheumatologist. Non-pharmacological interventions included were complementary and alternative physical therapies (e.g. transcutaneous electrical nerve stimulation (TENS), laser therapy, acupuncture, ultrasound), exercise (land-based and aquatic), nutritional therapy, orthotics management (corrective appliances/splints), and thermotherapy [28].

Recommendations were presented through algorithms for the diagnosis, early management and long-term monitoring of JIA. Once a diagnosis of JIA has been made, it is suggested to commence initial management of disease using short term analgesics (e.g. acetaminophen) and an NSAID [28]. When symptoms of inflammation continue for more than 4 weeks, patients should be referred to a pediatric rheumatologist for advanced treatment (GC intra-articular injection, oral or intravenous corticosteroids, DMARDs, and biologics) [28]. In the 6 to 12 weeks after diagnosis, basic pharmacological treatment (simple analgesics such as acetaminophen, NSAIDs and weak opioids) can be provided by the GP, while consulting a pediatric rheumatologist [28]. Beyond 12 weeks, the GP should monitor the child or adolescent and refer them to the appropriate multidisciplinary healthcare professionals (e.g. physiotherapists, occupational therapists, dieticians, podiatrists) who can provide non-pharmacological interventions as an additional component of a patient’s care plan. GPs are also advised to manage acute flares of JIA using analgesics with or without NSAIDs and to involve a pediatric rheumatologist and other healthcare professionals in the event that disease flares continue [28].

GKJR 2012.

The GKJR CPG was intended for a variety of individuals, including physicians and other healthcare professionals who care for patients with uncomplicated JIA [27]. Multidisciplinary treatment is recommended, including both pharmacological and non-pharmacological interventions. The types of pharmacological interventions considered were NSAIDs, GCs (intra-articular injection and systemic), DMARDs, and biologic agents (tumor necrosis factor alpha (TNFα) inhibitors, interleukin-1 (IL-1) inhibitors, interleukin-6 (IL-6) inhibitors, and co-stimulatory antagonists). Non-pharmacological interventions included physiotherapy and occupational therapy, low-impact sports and exercise, nutritional therapy, surgery, and psychological and socio-pedagogical care [27]. The pharmacological recommendations for systemic GCs, DMARDs and biologics sometimes specified if they were for oligoarticular JIA, polyarticular JIA, or sJIA; otherwise, the recommendations were general. The CPG did not discuss how to treat those individuals who were resistant to medications or obscure and difficult arthritis cases [27].

The clinical recommendations were presented for each JIA onset type. For oligoarticular JIA, treatment starts with an NSAID, then intra-articular injections (with GCs) are added and finally DMARDs (specifically methotrexate (MTX)). For polyarticular JIA, the initial treatment is a combination of an NSAID plus GCs (intra-articular injection or systemic). As an escalation of treatment, the next step is to treat with a DMARD (specifically MTX). Finally, if disease activity is high, a TNFα inhibitor should be added to the treatment, but if disease activity is low, the DMARD should be replaced with a TNFα inhibitor. For sJIA, NSAID and systemic GC (high dose or pulse) interventions are initially recommended, followed by the addition of DMARDs (specifically MTX) as an escalation in treatment. Finally, a biologic agent should be added if disease activity continues. For each onset type, physiotherapy interventions should be used concomitantly throughout the escalation of pharmacological interventions according to GKJR in order to ensure a multidisciplinary approach to care [27].

ACR 2011 & 2013.

The ACR CPG was aimed toward pediatric rheumatologists to give recommendations on NSAIDs, GCs (intra-articular injection and systemic), DMARDs, and biologics [29,30]. The recommended pharmacological intervention depends on whether disease activity is low, moderate, or high and if an individual has features of poor prognosis or not [29,30]. The 2011 ACR CPG focused on the most frequently used medications in each drug class, while the 2013 ACR CPG update included medications specific to treating sJIA [29,30]. There was no discussion of contraindications, intolerance, tapering or discontinuing medications in the event of inactive disease [29]. Instead of presenting the patient populations of interest according to JIA onset types, ACR used the following clinical scenarios: 1) history of arthritis in 4 or fewer active joints, 2) history of arthritis in 5 or more active joints, 3) sJIA with active systemic features but no active arthritis, 4) sJIA with active arthritis but no systemic features. Other clinical scenarios considered were active sacroiliac arthritis in the 2011 CPG and sJIA with features of macrophage activation syndrome (MAS) in the 2013 update [29,30].

Features of poor prognosis varied according to clinical scenario and were clearly defined in tables in the CPG [29]. For those with a history of 4 or fewer joints at least one of the following features of poor prognosis had to be satisfied: arthritis of the hip or cervical spine, 2) arthritis of the ankle or wrist plus a noticeable or extended time with elevated inflammatory markers and 3) radiographic damage (erosion of joint or reduction in joint space) [29]. For those with a history of 5 or more joints, the features are the same except for the second one, which is positive rheumatoid factor or presence of anticyclic citrullinated peptide antibodies [29]. The clinical scenario of systemic arthritis with active arthritis but without active systemic features has only two features of poor prognosis: arthritis of the hip and radiographic damage (erosion of joint or reduction in joint space) [29]. For sJIA with active systemic features but without active arthritis the features of poor prognosis were defined as: 1) systemic disease being active for at least 6 months (fever, elevated inflammatory markers), or 2) need for systemic GC intervention [29]. The 2013 update used active joint count and physician’s global assessment to assist in determining the need for specific interventions rather than the features of poor prognosis [30].

Each clinical scenario has described disease activity levels being low, moderate, or high. There were specific requirements for each scenario, but common among them for low disease activity was having a normal erythrocyte sedimentation rate or C-reactive protein level and receiving a score below 2/10 on the patient/parent global assessment of overall well-being [29]. For moderate disease activity, each clinical scenario required that patients experience one or more low disease activity criterion, but under three that were considered high disease activity [29]. To be classified as high disease activity, among other specific criteria of each clinical scenario, patients must have a doubled erythrocyte sedimentation rate or C-reactive protein level compared to what is considered the upper limit of normal and have a score of 7 or higher out of 10 on the physician global assessment of overall disease activity. Unlike with low disease activity where all criteria must be satisfied, all but one of the criteria for a particular clinical scenario could be satisfied. Systemic arthritis with active systemic features but without active arthritis had two levels of disease activity, depending on the physician global assessment of overall disease activity (above or below 7/10) [29].

The ACR CPGs [29,30] made separate treatment algorithms for each of their clinical scenarios. For a history of 4 or fewer joints, NSAID monotherapy is given initially to those with low disease activity without features of poor prognosis (FPP) and to start on GC injections after 2 months if the disease activity (DA) remains or worsens. For any other combination of features of poor prognosis (FPP) and disease activity (DA), the initial treatment is GC intra-articular injections with NSAIDs as necessary. The escalation of treatment would lead to MTX as the next step for any of these patients (with NSAIDs or GC intra-articular injections as necessary) and then finally a TNFα inhibitor (with NSAIDs or GC intra-articular injections as necessary). Among individuals with a history of 5 or more joints, MTX is recommended (with NSAIDs or GC intra-articular injections as necessary) when there is high disease activity (DA) or moderate disease activity with features of poor prognosis or after 1 month of NSAIDs for low disease activity and features of poor prognosis or moderate disease activity and any prognosis [29]. After MTX treatment for either 3 or 6 months, depending on disease activity, a TNFα inhibitor (with NSAIDs or GC injections as necessary) should be given. Finally, if after 4 months the disease activity is still moderate or high then a different TNFα inhibitor or abatacept should also be prescribed in the place of the first one [29]. For systemic arthritis with active systemic features, NSAIDs should be the initial intervention only if the physician’s global assessment is less than 7 and the patient presents with fever, but has no features of poor prognosis. Otherwise they should be started on systemic GCs (with NSAIDs as necessary). If fever continues, the treatment is escalated to anakinra (with GC intra-articular injections or NSAIDs as necessary). The treatment pathways are more complex in the 2013 update and the pathways depend on the active joint count for determining the disease activity. With a lower joint count (no more than 4) initial intervention will be either NSAID monotherapy or GC intra-articular injections, but MTX or leflunomide will be the initial intervention when more than 4 joints have active disease [30].