Anticoagulation Knowledge Tool Development

We began by conducting a comprehensive review of the literature on patient anticoagulation knowledge, with additional information obtained from freely available patient educational material. The knowledge domain covered in the review of the literature included basic drug information, adverse drug effect, drug-drug interactions, drug monitoring and dietary issues. Similar information was then grouped to form a list of 56 items consisting of both open ended and multiple choice questions. The usefulness of each question in assessing anticoagulation knowledge was then discussed by the authors, after which the items were ranked on a scale of 1 to 5 (1 = strongly disagreed, 5 = strongly agreed) in terms of their relevance to anticoagulation knowledge. These rankings were used to eliminate irrelevant questions and create a 28-item draft instrument.

The items in the draft instrument were then discussed with 15 selected people from a non-medical background to ensure clarity of the sentences, simplify wording and to identify ambiguous and misleading terms. Items in the draft instrument were reworded based on the feedback received.

Content Validity

Content validity refers to the degree to which a scale has an appropriate sample of items to represent the construct of interest [11]. To ensure content validity, the draft instrument was presented to 10 anticoagulation experts (8 pharmacists and 2 physicians) selected based on their work experience or research related to the use of oral anticoagulants. These experts were asked to rate the relevance of each item on the draft instrument on a four-point ordinal scale (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant), and to suggest other items for the scale which may have been omitted. The content validity index for each item (I-CVI) and overall content validity of the scale (S-CVI) was then calculated using the method of Polit et al [11, 12]. In calculating the I-CVI, the rating scale was dichotomized, with ratings of ‘1’ and ‘2’ combined as not relevant, and ratings of ‘3’and ‘4’ combined as being relevant, while the S-CVI was calculated by determining the average of all the I-CVI values. Further, I-CVI values were translated into values of a modified kappa index (k*) so as to adjust for chance agreement among the experts participating in the content validity exercise. The modified kappa index was determined using the formula–k* = (I-CVI- pc)/ (1- pc), where pc refers to the probability of chance agreement among the experts and was computed using the formula for a binomial random variable, with one specific outcome (pc = [N! /A! (N—A)!] x 0.5^N; where ‘N’ = number of experts and ‘A’ = number of experts agreeing on relevance of an item). The average S-CVI of the scale was 0.92 with I-CVIs ranging from 0.6–1 and k* ranging from 0.5–1 (Table 1). The final instrument was divided into two sections–section ‘A’ and ‘B’, with section ‘A’ comprising general anticoagulation knowledge questions applicable to both the DOACs and VKAs, and section ‘B’ comprised of VKA-specific questions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Item and Scale Content Validity Indexes.

https://doi.org/10.1371/journal.pone.0158071.t001

Pilot Study

In order to further ensure readability and comprehension, a pilot study was conducted in 13 participants (5 pharmacists, 3 patients and 5 members of the general public) representing the three groups to be compared. The results from the thirteen pilot studies participants were not included in the main study. Instructions on completing and returning the questionnaire were further revised based on the feedback obtained in the pilot study. The final instrument used in the study is available in S1 Appendix.

Validation Study

Adults (aged > 18 years) who were able to read and complete the questionnaire independently were recruited into the validation study. All the participants in the validation study were recruited from Tasmania, Australia. Subjects were recruited into three groups comprising of a pharmacist (expert) group, patient group and general public group. The pharmacist group was expected to serve as the positive control while the general public group was expected to serve as the negative control. Pharmacists were recruited from a total of 26 community and hospital pharmacies; patients currently prescribed oral anticoagulants were recruited from 14 community pharmacies; and participants from the general public group were recruited from 12 public places (e.g. parks, bus stops and shopping malls). Participants from the general public group were eligible to participate in the study if they were not health professionals, patients prescribed oral anticoagulants and did not have close relationships with patients taking oral anticoagulants. A study information sheet for the study was provided to participants in the three groups which stated that anticoagulants are also called blood thinners, specifically to assist participants in the general public group who may be less familiar with the term ‘anticoagulant.’ Also, written informed consent was obtained prior to participation. Participants in the pharmacist and general public group were required to assume that they were currently taking an oral anticoagulant and answer the questions in both sections of the survey, while participants in the patient group were asked to respond to the survey based on the oral anticoagulant they had been prescribed by their physician. Patients who were prescribed any of the DOACS were required to answer the questions in section ‘A’ only, while patients who had been prescribed VKAs were asked to answer the questions in both sections. Participants in the pharmacist group were given the option of completing the test online or by using a paper format, while the other two groups completed the test by using only the paper format. Participants who preferred to use the paper format had the option of completing the survey on the spot, or return it using a reply paid envelope. The study protocol was reviewed and approved by the Tasmanian Health and Medical Human Research Ethics Committee.

Validity and Reliability

Construct validity refers to the extent to which a measure adequately assesses the construct it purports to assess [13]. Construct validity was assessed using the contrasted group approach which involves identifying two or more groups of individuals who are expected to have different scores on the characteristics being measured by an instrument [13]. Using this approach, we hypothesised that the instrument would be sensitive to multiple levels of anticoagulation knowledge. Also, we expected the mean score of the pharmacist (expert) group to be higher than the mean patient group score, and the mean score of the patient group to be higher than that of the general public group.

Two reliability tests were conducted: test-retest reliability and internal consistency reliability. In order to ensure the instrument’s stability, a re-test was conducted at approximately 2–3 month after the initial test administration, a time period considered sufficient to reduce the impact of recall. All the participants in the pharmacist and patient group were eligible for re-test, with 32 participants in the patient group and 22 in the pharmacist group participating in the second test. Internal consistency reliability was also conducted across the three groups to ensure the inter-relatedness of the items in the instrument.

Scoring

Scoring was done use a dichotomous scale, with a score of ‘1’ or ‘0’ for each correct answer or wrong answer, respectively. A maximum score of ‘1’ was allocated to each correct answer for all of the questions with the exception of item ‘6’, ‘18’ and ‘19’ in section ‘A’ and item ‘6b’ in section ‘B’. A maximum score of ‘2’ was obtainable for item ‘6’ in section ‘A’- (‘Why is it important to take this medicine exactly as your doctor has told you?’) - 1 mark each was allotted for answers related to the prevention of thromboembolism and answers related to minimising the risk of bleeding. For items ‘18’ and ‘19’- (‘three signs of side effects you should watch out for’ and ‘three things you can do to reduce your risk of side effect’, respectively) - 1 mark each was allotted for each correct sign of side effects to look out for and each correct approach to reduce the risk of bleeding. Lastly, for item ‘6b’ in section ‘B’ (‘list three foods that can affect your anticoagulant therapy’) - 1 mark each was allotted for three correct food substances mentioned. A maximum total score of ‘25’ was obtainable for patients taking the DOACs required to answer only section ‘A’ of the questionnaire, while a maximum total score of ‘35’ was obtainable for patients taking the VKAs (warfarin) required to answer both sections of the questionnaire. Final scores were presented as a percentage of correct answers for all the participants in the study.

Statistical Analysis

Analysis of variance (ANOVA) was used in comparing the mean scores between the pharmacist, patient and general public groups, with p<0.05 considered statistically significant. Pearson's correlation was used in determining the correlation between the test and re-test scores for the pharmacist and patient groups, and a values between 0 and 0.49 were considered as ‘very low’ to ‘low’ correlation, while values between 0.5 and 1.0 were considered as ‘moderate’ to ‘very strong’ correlation. Cronbach’s alpha score was used in determining internal consistency reliability and across the three groups, with a score of 0.7 or greater considered acceptable [14]. Lastly, the relative difficulty of each item and the instrument’s ability to discriminate between groups was also analysed by determining the differences in the percentages of items correctly answered across the three groups. Statistical analysis was conducted using SPSS Version 22.0.

Limitations

Among participants in the general public group, about 70% had formal education beyond high school level, including 28% with a post-graduate qualification. The high literacy level of this group may not be truly representative of the general public. However, the average score of this group was still significantly lower than both the patient and pharmacist groups. Also, participants in the three groups were not aged matched, and it is not known if a higher median age in the general public group would have given a higher result. All the participants in the survey were given the opportunity of either completing the survey immediately upon receipt or returning it using a reply paid envelope; we cannot rule out the possibility that some participants might have accessed additional resources despite being encouraged not to do so in the survey instructions. This may have increased the overall score in the survey. The relatively high score in the patient group may be as a result of the recruitment of confident and enthusiastic patients who have had or are undergoing some form of educational training on the use of oral anticoagulant medication, and may not necessary reflect the broader anticoagulant-medication taking population. For the test-retest reliability, not all the participants who completed the first test participated in the second test, and the impact of this on the test-retest reliability coefficient remains unknown. Lastly, the study was conducted in a single region, and the instrument may need to be validated in other regions globally.