Participants

Six group co-ordinators were recruited (4 female, 2 male), each of whom led one of the 4 intervention groups (two groups had paired co-ordinators working in a job share). All had at least 3 years’ experience, (mean 4.7. Range 3–7) in leading community groups for people with aphasia. One was a qualified speech and language therapist (SLT). Co-ordinators, and their associated groups, were geographically dispersed. For example, one was in the North of England and one was in the South. Groups were designated ‘North’, ‘South’, ‘East’ and ‘West’ according to their location.

Eleven volunteers were recruited (6 females, 5 males), with at least two assigned to each intervention group. Note that this was below our protocol target of 4 per group. Some had long-standing links with the co-ordinator and their respective community setting. Others were newly recruited for this study. All had previous experience in working with people who have aphasia. Four were experienced group volunteers, three were student speech and language therapists (SLT) and one was a qualified SLT. Three were stroke survivors, two of whom had mild aphasia.

Thirty-four people with aphasia caused by stroke were recruited by the project managers (Authors ND, RT). The selection criteria were as follows: moderate or mild aphasia; no co-existing diagnosis affecting cognition; no severe hearing or visual impairments; fluent in English before their stroke. Eligibility was established by administering the Frenchay Aphasia Screening Test [45] and a brief screening questionnaire, e.g. asking about the history of the stroke, language background and any co-morbidities. The criteria aimed to ensure that individuals would be able to participate in the intervention. For example, communication in EVA Park is largely speech dependent making it difficult to access by people with severe impairments in the production or comprehension of speech. Severe cognitive and sensory impairments also impede use of this technology. Computer competence was assessed with a Dynamic Assessment of Computer Learning [46, 47]. Here the individual was required to carry out a simple instruction using a laptop or tablet computer, such as consulting an internet weather report. If unable, they were provided with a hierarchy of cues, including verbal instructions and modelling, and their ability to respond to these cues was assessed. This ensured that the participant either had basic computing skills or could develop these with support. Prior computer usage was not an exclusion criterion.

Participants were either existing members of the co-ordinators’ community groups or were assigned to groups on the basis of their geographical location. This aimed to ensure that relationships formed during the online intervention might be sustained in the real world after that intervention ceased.

The sample size was informed by recommendations for feasibility studies which call for between 24 and 50 participants [48, 49]. The size of each intervention group was also constrained by the number of people who could be effectively managed in EVA Park. As this was a feasibility study, the sample was not powered to provide definitive data about the benefits of the intervention.

Participant recruitment began on 16.5.2017 (date of first screening appointment). Data collection ended on 17.11.2018 (date of final follow up assessment).

Intervention

There were four intervention groups, each led by at least one co-ordinator and 2 volunteers, and each involving between six and nine people with aphasia. Intervention comprised 14 group sessions (21 hours) delivered over 6 months, with sessions occurring once a fortnight (see S1 Supporting Information in S1 File for an outline of sessions and sample session plans). All sessions were run remotely, in EVA Park. Thus participants accessed the intervention on a computer in their own home. Co-ordinators and volunteers worked either from a home computer or from a computer in their community centre. All participants were represented by personalised avatars in EVA Park, which were set up before the start of intervention.

The intervention was defined in a manual that drew on published accounts of support interventions for people with aphasia [e.g.: 50, 51]. The aims and content of the intervention were also informed by discussions with our Advisory Group, comprising four people with aphasia, a family member and three voluntary sector staff members who were responsible for managing and delivering stroke services. Intervention aimed to counter the negative impacts of aphasia on quality of life, and to facilitate living well with aphasia [6]. Activities aimed to promote wellbeing, give participants experiences of communicative success and foster social connection between group members. Group members frequently reflected on personal strengths, and how these were applied to living with aphasia. Issues of personal identity were also focussed, given the impact of aphasia on a person’s sense of self [5, 51].

Each intervention session was based on a topic, which was chosen with the Advisory Group to address the intervention goals. Several topics enabled group members to share experiences of living and coping with aphasia (e.g. ‘You’; ‘Aphasia’; ‘Resilience’; ‘Personal Strengths’). These sessions drew on the exercises described by Holland [52] that foster adaption, growth and the development of positive attitudes concerning aphasia and its consequences. For example, in the session on ‘Aphasia’ participants discussed their residual communication strengths and the strategies that they employed to ease communication.

Other topics aimed to stimulate social connection and positive communication exchanges. They provided opportunities for group members to express opinions and convey aspects of their personality, thus addressing the theme of identity. These topics included ‘Comedy’, ‘Music’, ‘Art’, ‘Literature’ and ‘Eating Out’. To illustrate, the Comedy session involved a discussion about the benefits of comedy for wellbeing, sharing views about comedy, reacting to a comedy video clip (displayed on a media screen in EVA Park) and sharing personal stories about funny events.

All sessions aimed to give participants the experience of communication success. Formally correct language was not demanded and total communication devices were encouraged, e.g. employing alternatives to speech [53]. These devices included tone of voice, message writing and demonstration. The EVA Park avatars were able to execute a small number of pre-programmed gestures (such as waving or an exaggerated belly laugh), which were performed when a screen icon was clicked. These were used to supplement communication, for example to express appreciation over a joke.

Another thread that ran through the intervention was the identification of personal strengths, and reflection on how participants used these in their daily lives. This was in line with the principles of positive psychology and asset-based interventions [54–56]. For example, each session included a review activity at the end in which participants were asked to identify ‘three good things’ that had taken place. Applying personal strengths in the context of meaningful activities was particularly addressed in two ‘Project’ sessions. Here, group members chose to create something collectively (e.g. an aphasia awareness film, or an audio podcast about aphasia). This encouraged the group to work towards a shared goal, and to draw on their individual skills in realising that goal. For example, one former video editor used his skills in making a film about EVA Park.

Group leadership was provided by the co-ordinators. They introduced each topic and led the activities, for example by assigning roles and turns to group members. They ran group discussions, ensuring that each member had the opportunity to contribute. They managed transitions between activities. They closed each session with feedback to the group and by looking ahead to the following week. Volunteers contributed to group discussions and supported individuals’ communication attempts, for example by asking questions or providing cues. They supported small groups or pairs of participants in instances when the groups sub-divided.

Between session challenge tasks (e.g. to meet with another group member for exercises in the EVA Park Health Centre) aimed to provide further experiences of success and extend the social contacts made in the groups. These tasks required group members to access EVA Park independently, without the support of volunteers.

Training of the intervention providers

Before the intervention began, the project managers (ND, RT) provided the group co-ordinators and volunteers with two, 4-hour face-to-face training sessions. The first session covered technical aspects of group provision in EVA Park including how to set-up participants in EVA Park, avatar creation and customisation, navigation in EVA Park, troubleshooting and providing remote technical support to people with aphasia. The second session provided training on delivering the intervention described above. This included practice in managing a group in a virtual environment. Specific advice was given on how to employ supportive communication techniques in EVA Park. Such techniques included pointing, giving time, reflective listening, using message writing to support understanding, and using visual cues. In addition to training, the project managers provided monthly supervision sessions to the co-ordinators and volunteers from each intervention group (6 sessions per group, each 40 minutes long). These took place in a private area of EVA Park that was not accessible to all users. Sessions covered general feedback, attendance, technical issues and solutions, use of interactive elements and immersion in EVA Park. They also covered research administration such as record keeping and economic data collection. Support was also provided to co-ordinators and volunteers throughout the 6-month intervention period, through responses to telephone and e-mail queries.

Training and set up of intervention recipients

The co-ordinators set up and trained their group participants, through individual home visits. Set up involved downloading the browser that ran EVA Park (Firestorm) onto their computer, adding the EVA Park web address, and setting up the participant’s avatar. Training followed a 20 minute protocol and taught each participant how to log in; make their avatar walk, fly and sit; turn on their microphone; adjust the volume of other users and adjust their camera angle.

EVA Park required an internet connection of at least 2MB per second download and 500KB per second upload. Recommended ‘standard’ computer requirements were: 1GHz processor, 1GB memory and a dedicated graphics card (such as Nvidia graphics cards 6600 or better). Twenty participants had hardware that did not meet these requirements. They were, therefore, loaned a computer for the duration of their intervention.

Design

The study employed a randomised, waitlist controlled design. Two intervention groups were randomised to run in months 6–11 of the study (immediate condition). The other two groups ran in months 13–18 (delayed condition). Randomisation took place before any participants with aphasia were recruited and was conducted using a computer randomisation tool (the list randomiser in www.random.org). Those who delivered the intervention (co-ordinators and volunteers) and those who received intervention (the participants with aphasia) were not blinded to condition. Those who administered and scored the outcome measures were blinded (see below). Participants only had access to EVA Park during their intervention period. Usual care continued for all participants with aphasia for the duration of the study.

Feasibility measures

The number of participants screened, recruited and retained during the study was recorded, together with reasons for non-recruitment and attrition. Records of attendance at intervention were kept. Automatic computer logging recorded the number of times EVA Park was visited and the amount of time spent in EVA Park by each participant. The number of adverse events experienced by participants was recorded.

We aimed to record 30 intervention sessions using screen capture technology (3 recordings failed owing to technical difficulties). Sampling was spread across the groups and ensured that sessions were filmed from early and late weeks of the intervention. The sampling regime was drawn up blind to the content of intervention and before sessions were run. Eighteen of the recorded intervention sessions (just over 32% of all intervention sessions) were subject to fidelity checking. A fidelity checklist was developed with reference to the intervention manual and in discussion with the project managers (ND & RT). This reflected core features of the intervention, or elements that were essential to the underlying principles of the intervention (see S2 Supporting Information in S1 File). The recorded sessions were reviewed, and each feature on the checklist was scored as fully present (2), partially present (1), absent (0) or not applicable. The fourth category covered instances in which there was no opportunity for a feature to arise in a session. Fidelity scoring was conducted by two researchers who were not involved in delivering the intervention or in any other aspect of this study (Authors GM, KM). Both were qualified speech and language therapists. Five sessions were scored independently by both researchers to assess inter-rater reliability. As there were 24 items on the fidelity checklist this yielded 120 double coded treatment components.

Outcome measures

Two primary outcome measures were identified:

Warwick-Edinburgh Mental Well-being Scale (WEMWBS) [57]. This scale was developed to monitor mental wellbeing in the general population and evaluate intervention effects on mental wellbeing. It comprises 14, positively worded statements, such as ‘I’ve been feeling good about myself’, that have to be rated on a 5 point scale. These are summed, yielding a single wellbeing score (/70).

Communication Activities of Daily Living-2 (CADL-2) [58]. This is a standardised assessment of everyday communication for people with aphasia, which is based on specific scenarios, such as going to the doctor. The CADL-2 contains 50 items scored 2, 1 or 0. Scoring credits communicative success rather than the use of formally correct language. The total score was analysed; i.e. the score across all 50 questions (/100).

Secondary measures were:

Social Connectedness Scale-Revised (SCS-R) [59]. This consists of 20 positively or negatively worded statements, such as ‘I feel close to people’ and ‘I see myself as a loner’, that have to be rated on a six point scale. All items are summed (with reverse scores for negative items) to produce a single score (/120).

Western Aphasia Battery-Revised (WAB-R) [60]. This is a standardised language assessment designed for people with aphasia. Only sections assessing speech production and comprehension were administered. These produce a single aphasia quotient score (/100) that was analysed here.

Stroke and Aphasia Quality of Life-39 (SAQOL-39g) generic version [61]. In this assessment 39 items, such as asking how much trouble someone had speaking in the last week, must be rated on a five- point scale. Three domains are covered: physical, psychosocial and communication, yielding mean domain and total scores. The mean total was analysed here (/5).

All measures have strong psychometric qualities and have shown sensitivity to therapy induced change. Three (CADL-2 [58], WAB-R [60] and SAQOL-39g [61]) have been widely used in previous trials of aphasia therapy [62], and two (SAQOL-39g [61] and WAB-R [60]) have since been advocated as core outcome measures, for use in all aphasia therapy trials [63]. Two measures (WEMWBS [57] and SCS-R [59]) were not designed and have not been validated for people with aphasia. However, they are brief and make limited language demands. The WEMWBS [57] has been used in a trial of peer support for people with aphasia [64]

The outcome measures were administered to participants with aphasia in months 5 (T1), 12 (T2) and 19 (T3). Thus, participants in the immediate condition were tested before intervention, after intervention and at 6 months follow up. Those in the delayed condition were tested twice before intervention began and once after intervention. Testing was conducted face to face, not in EVA Park. Participants were tested in their own homes, or in their local community group setting. Testers were not involved in any other aspect of the study, and were blinded to time point and condition. Testers were qualified or student speech and language therapists who were trained in test administration.

Mean scores, standard deviations (s.d.) were calculated at each time point. For skewed variables, medians and interquartile ranges (IQR) were also calculated. The primary analysis employed ANCOVA to compare assessment results at T2 across participants in the immediate and delayed conditions, with scores at T1 as the covariate. Here, indicative treatment effects would be demonstrated by a significant group effect favouring those in the immediate condition. This analysis was performed on both an intention to treat (ITT) and per protocol basis. The former used carried forward scores for any missing data. A secondary, per protocol, analysis examined pre to post intervention change across all participants who completed the intervention and for whom outcome measure data were available.

Economic data collection

A comprehensive and detailed approach was taken to assess economic costs for each group from a provider perspective. This entailed (a) identifying all key inputs associated with the intervention, (b) devising approaches and instruments to measure such inputs separately for each group, (c) estimating the costs associated with all inputs separately for each group, (d) estimating total costs separately for each group and average total costs across the four groups and (e) examining variations in costs across the four groups.

The following key input components were identified and then measured either prospectively (by researchers) or retrospectively (by coordinators and volunteers) using a combination of specifically designed forms issued online and routine project records:

• resources and expenses associated with the project managers delivering training for coordinators and volunteers and supporting the running of the groups;
• resources and expenses associated with service coordinators attending training and contributing to the running of the groups;
• resources and expenses associated with volunteers attending training and contributing to the running of the groups;
• loaning of computer hardware to participants; and
• set-up and support for the computer software.

All coordinators and volunteers completed and returned the requested data. Expenses were taken as reported or estimated from external sources as necessary. Hardware and software costs were appropriately apportioned to groups. For example, the hardware costs reflected the number of computers that had to be loaned per group, while software costs reflected a share of the server and technical support costs. Time inputs were valued by multiplying them with appropriate unit costs (see S3 Supporting Information in S1 File). Coordinator time was valued at £30 per hour, volunteer time at £6.86 per hour and researcher time at £66 per hour. All costs were standardised at 2017/18 price levels and discounting was unnecessary because all costs related to a period of less than one year. Costs are presented in British pounds sterling (£), where £1 is equivalent to approximately 1.466 United States dollars (USD) based on 2017 purchasing power parity rates [65]

Costs for the various components were then summed to generate the following summary totals:

• Total cost per group and the average of this across groups (including and excluding hardware since more equipment loaning than anticipated became necessary to ensure participants had requisite standard computer hardware and it is unclear to what extent this would be replicated elsewhere)
• Average cost per participant per group, accounting for actual participant numbers observed for each group, and the average of this across groups (exc. hardware)
• Average cost per scheduled online attendance per group, accounting for actual total attendance observed for each group, and the average of this across group (exc. hardware)

RQ1 feasibility results

Recruitment and retention (see Fig 1).

Thirty four people with aphasia were recruited over a period of 13 weeks, out of 67 screened. The consent rate of those eligible to take part was 72.34% (34/47). Twenty-two participants were recruited through the four community aphasia groups led by the co-ordinators. Three were recruited via self-referral and nine participants were referred via other community groups.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Study flow diagram.

https://doi.org/10.1371/journal.pone.0239715.g001

Of those recruited, 31 began the intervention (91.18%). Twenty nine (85.3%) completed intervention and 27 (79.4%) completed all assessments. Reasons for attrition were family/health reasons (2); participant obtained employment (1); travel abroad (1); participant recruited by another research project which forbad involvement in this study (2); participant opted to withdraw (1). The total time from the first recruitment to the final T3 assessment was 18 months. The trial ended when data collection (minus attrition) was complete.

Participant characteristics are reported in Table 1. Data for age and months post stroke were not normally distributed. The conditions differed on two baseline characteristics. The delayed condition was older than the immediate condition (Mann-Whitney U = 77, p = .021) and contained more women (chi square (1) = 5.78, p = .016). The conditions did not differ with respect to months post stroke, presence of hemiplegia, educational attainment or the number who were in professional employment pre-stroke.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Participant baseline characteristics.

https://doi.org/10.1371/journal.pone.0239715.t001

Seven participants failed to complete intervention and/or all assessments (see Fig 1). These participants did not differ from those with full data on any baseline characteristics (age, p = .6; months post stroke, p = .91; gender, p = .4; hemiplegia, p = .6; education, p = .8; employment, p = .4); although it should be acknowledged that the power to detect a difference was low. Two participants experienced adverse events during the trial, both of whom were in the delayed condition. These events were a hospital admission and referral to GP for depression.

Fidelity.

Inter-rater reliability of the fidelity checking scores was excellent. One hundred and twenty treatment components were double coded across five videos, with 96.7% agreement, Kappa = .92, p < .001.

Across all 18 videos that were evaluated, the mean fidelity score per treatment component was 1.76 (/2; range 1.45–2; median 1.82). In these videos 81.9% of the applicable treatment components were fully present (scoring 2), 12.6% were present to some degree (scoring 1) and 5.5% were absent (scoring 0). Ten videos were taken from the first 5 sessions of intervention, and 8 from the last 5 sessions. Fidelity scores did not differ for early and late sessions (early mean score = 1.758; late mean score = 1.761; t (16) = -.039, p = .97). Videos were also sampled from each intervention group. Their scores are reported in Table 2 and were not significantly different (Kruskal-Wallis H (3) = 5.16, p = .16).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Mean treatment fidelity scores across intervention groups.

https://doi.org/10.1371/journal.pone.0239715.t002

RQ2 outcome measures

Scores on all outcome measures at the three time points are reported in Table 3. The primary ANCOVA analysis compared scores at T2 with scores at T1 as the covariate. This was first performed on an ITT basis. Therefore, three T2 scores were imputed (two in the immediate condition and one in the delayed condition), by carrying forward the last data point. T3 data is reported on a Per-Protocol basis. It includes all participants who completed the intervention and the T3 assessments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Mean (sd) [Median; IQR] scores on the outcome measures at T1, T2 and T3; treatment effects in the primary analysis.

https://doi.org/10.1371/journal.pone.0239715.t003

Preliminary checks were conducted on the T1 and T2 data to ensure that there was no violation of the assumptions of normality, linearity, homogeneity of variance, homogeneity of regression slopes, and reliable measurement of the covariate. These assumptions were met with all the data except CADL-2 scores at T1, where data in both conditions were not normally distributed (Shapiro Wilk p < .05). However, as the skewness value was only just outside the acceptable range (skewness = -1.11) the ANCOVA was conducted.

After adjusting for T1 scores, there were no significant differences between the immediate and delayed conditions at T2. The only measure that approached significance was the WAB-R (p = .051) with a medium effect size (partial eta squared = .12). For all measures there was a strong relationship between T1 and T2 scores (partial eta squared range = .55 - .82). The ANCOVA analyses were repeated on a Per Protocol basis (immediate n = 14; delayed n = 17), with no difference in the findings. The WAB-R result was again just short of significance (p = .051; partial eta squared = .13). Therefore, both the ITT and Per-Protocol ANCOVA analyses failed to detect any indicative effects of therapy on these measures.

Table 4 reports pre and post intervention scores on all measures. Scores are reported for all participants who completed the relevant assessment points and the intervention. For participants in the immediate condition pre/post scores are drawn from T1 and T2; while for those in the delayed condition they are drawn from T2 and T3. All data were normally distributed except for the pre intervention CADL-2 data (Shapiro Wilk, p < .05).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Mean (sd) [Median; IRQ] scores for all participants on the outcome measures pre and post intervention; n = 28.

https://doi.org/10.1371/journal.pone.0239715.t004

Change on most measures was non-significant, and with small effect sizes. Only the WAB-R showed a significant improvement, with a medium effect size.

RQ3 cost estimates

Cost estimates are summarised in Table 5, with further details of each component described in S4 and S5 Supporting Information in S1 File. Total costs per group ranged between £6,516 and £11,316 ($9,552 USD and $16,589 USD) excluding hardware, and £7,483 and £12,562 ($10,970 USD and $18,416 USD) including hardware. The most costly component across all groups was the training for coordinators and volunteers at an average cost of £3,414 ($5,001 USD) per group.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Summary of costs.

https://doi.org/10.1371/journal.pone.0239715.t005

The different implementation contexts across the groups were reflected in the costs. For example, one group (East) required considerably less researcher time for the training compared with the other three because its training sessions took place at the project managers’ university base and thus incurred no travel time or expenses for the trainers (whose time was more expensive than that of coordinators and volunteers).

The average cost per participant across the four groups was £1,364 ($2,000 USD) (excluding hardware). This was naturally driven by group size as much as the input variations across the groups. The group with the lowest input costs had the largest group size so generated an average cost per participant of £724 ($1,061 USD) as compared with averages ranging between £1,143 and £2,263 ($1,676 USD and $3,318 USD) for the other three groups. Conversely, the group with the largest input costs also had the smallest group size, thus generating a much larger average cost per participant of £2,263 ($3,318 USD).

Future implications

Our findings indicate that it would be feasible to conduct a larger trial of remote group intervention for people with aphasia in EVA Park. However, the lack of any indicative treatment effects found in this study suggest that revisions should be made to the intervention and assessment of outcomes. Here, findings from our qualitative data will be crucial, but possible amendments include the use of a more intensive regime and smaller intervention groups. Other programmes of group social support have also involved family members [15], which might be considered for a future trial. Preliminary inspection of our qualitative findings has unearthed quite widespread difficulties with internet connectivity and sound quality, which need to be addressed before progression to a larger trial. Turning to assessment, the outcome measures should be reviewed, and possibly restricted to tools that were designed specifically for people with aphasia. The Assessment of Living with Aphasia might be considered as a primary outcome measure, given that this has shown treatment effects in a comparable study [36]. Our economic data can inform decisions about which components of the intervention are worth focussing on for a reliable assessment of costs. This should form part of a comprehensive economic evaluation, incorporating overall care costs for participants and relevant outcomes.