http://orcid.org/0000-0002-2703-7052
Figures
Abstract
Introduction
An acute shortage of senior mentors saw the Palliative Medicine Initiative (PMI) combine its novice mentoring program with electronic and peer mentoring to overcome insufficient mentoring support of medical students and junior doctors by senior clinicians. A three-phased evaluation was carried out to evaluate mentees’ experiences within the new CNEP mentoring program.
Methods
Phase 1 saw use of a Delphi process to create a content-valid questionnaire from data drawn from 9 systematic reviews of key aspects of novice mentoring. In Phase 2 Cognitive Interviews were used to evaluate the tool. The tool was then piloted amongst mentees in the CNEP program. Phase 3 compared mentee’s experiences in the CNEP program with those from the PMI’s novice mentoring program.
Results
Thematic analysis of open-ended responses revealed three themes–the CNEP mentoring process, its benefits and challenges that expound on the descriptive statistical analysis of specific close-ended and Likert scale responses of the survey. The results show mentee experiences in the PMI’s novice mentoring program and the CNEP program to be similar and that the addition of near peer and e-mentoring processes enhance communications and support of mentees.
Conclusion
CNEP mentoring is an evolved form of novice mentoring built on a consistent mentoring approach supported by an effective host organization. The host organization marshals assessment, support and oversight of the program and allows flexibility within the approach to meet the particular needs of mentees, mentors and senior mentors. Whilst near-peer mentors and e-mentoring can make up for the lack of senior mentor availability, their effectiveness hinges upon a common mentoring approach.
To better support the CNEP program deeper understanding of the mentoring dynamics, policing and mentor and mentee training processes are required. The CNEP mentoring tool too needs to be validated.
Citation: Krishna L, Tay KT, Yap HW, Koh ZYK, Ng YX, Ong YT, et al. (2020) Combined novice, near-peer, e-mentoring palliative medicine program: A mixed method study in Singapore. PLoS ONE 15(6): e0234322. https://doi.org/10.1371/journal.pone.0234322
Editor: Andrew Carl Miller, Nazareth Hospital, UNITED STATES
Received: December 22, 2019; Accepted: May 24, 2020; Published: June 5, 2020
Copyright: © 2020 Krishna et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data is within the paper.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Novice mentoring dominates mentoring in Palliative Medicine (PM) enhancing the personal, academic and professional development, clinical and professional practices and research productivity of mentees and mentors and boosting the reputations of host organizations [1–22]. Defined as a dynamic, context-dependent, goal-sensitive, mutually beneficial relationship between an experienced clinician and a junior clinician (or medical student), with the intention of supporting the development of the mentee, novice mentoring’s success is built upon the provision of timely, specific, appropriate, holistic, accessible and longitudinal support to nurture personalised mentoring relationships [21]. However, limited resources, variations in mentoring approaches and a shortage of trained and experienced mentors have jeopardized mentoring processes [23–40] and raised the possibility of ethical concerns in mentoring [41–64].
To circumnavigate some of these concerns, a combination of near-peer or peer and e-mentoring to supplement the prevailing novice mentoring approach is proposed. Lim et al. [65] define near-peer or peer mentoring (NP mentoring) as a “voluntary collaboration between colleagues of similar rank and experience and common academic interests on mutually beneficial structured fixed term projects. These processes often include a senior clinician who facilitates discussions, provides personalized support and feedback and oversees the mentoring process. Effective NP mentoring nurtures long-term friendships and professional collaborations between peers”. Chong et al. [66] define e-mentoring as a “personalised, internet or electronically mediated approach that is largely used to complement face-to-face mentoring to provide personalised, appropriate, specific, timely, holistic, accessible and longitudinal mentoring support to build mutually beneficial mentoring relationships between the host organization, a senior mentor and an individual mentee. Working within the confines of prevailing professional codes of conduct and standards of practice this approach is focused upon realizing the goals and needs of the mentee, the mentor, the host organization that supports and oversees the program and their relationships. Its asynchronous nature also nurtures reflective practices that helps develop deeper mentoring relationships”. Whilst Chong et al. [66]’s systematic review of e-mentoring, Tan et al. [5]’s systematic review of peer and near peer mentoring and Lim et al. [65]’s proposal for the use of a combination of novice mentoring and peer and near-peer mentoring, suggest that supplementing novice mentoring with near peer and e-mentoring will provide better oversight and timely, appropriate, personalised, specific, holistic and longitudinal support of mentoring relationships and mentees, combining novice, near peer/peer and e-mentoring to create Combined Novice, E-mentoring and Peer mentoring or CNEP mentoring remains untested [3, 32, 38–40, 67, 68].
However in the face of a sudden shortage of mentors following the untimely demise of a senior mentor and the sabbatical by another senior mentor, the Palliative Medicine Initiative (PMI), a research-based novice mentoring program hosted by the Division of Supportive and Palliative Care (DSPC) at the National Cancer Centre Singapore (NCCS), had to put theory to the test [21]. Building on data from Chong et al. [66]’s, Tan et al. [5]’s and Lim et al. [65]’s reviews the PMI established a CNEP program to maintain mentoring support for the 28 medical students affected by this sudden loss of mentors.
The PMI’s CNEP program
The PMI program was designed to increase student-led research in palliative medicine, professionalism, medical ethics, medical education, end-of-life ethics (EoLE) and health services through use of a novice mentoring approach. To do so, the PMI provided mentoring support to medical students and junior doctors through the various stages of the research and publication process. This homegrown approach was designed on the team’s own evaluation of prevailing data which later formed the basis for systematic reviews and scoping systematic reviews around the key elements of mentoring relationships [3], mentoring structure [6] and the mentoring environment [19].
By inculcating new novice [4, 7, 9] mentoring data, the PMI has evolved over time. However, at its centre is a flexible structure, approach and duration dependent on the specific research objectives, mentee’s abilities, motivations, goals and availabilities, the mentor’s goals, ability to support the mentee, experience and training, the progress of the research and mentoring project and the publication process. This process often ran between 6 months to 3 years in duration depending on the time taken to complete the publication of the research project in a peer reviewed journal. In Krishna et al. [21]’s account of the PMI program, there was a significant number of mentoring relationships that continued long after the completion of the primary research project with many mentees taking up new research projects under the PMI program.
This mix of flexibility and consistency within the mentoring approach has seen the PMI program publish over 50 mentored articles in peer reviewed journals and the presentation of more than 70 mentored posters in international medical education, medical ethics and Palliative Medicine conferences over the last 9 years [21].
The new CNEP mentoring approach supplements the PMI’s prevailing novice mentoring program with trained near peer-mentors who had successfully completed the PMI program. CNEP mentees and near peer mentors were also provided with access to e-mentoring options such as email, WhatsApp® (Facebook, Inc., Menlo Park, California, United States of America), text messaging, Facetime® (Apple Inc., Cupertino, California, United States of America) and Skype® (Microsoft Corporation, Redmond, Washington, United States of America) meetings with the senior mentor to ensure timely, specific, appropriate, personalised, longitudinal, accessible and holistic support and feedback. A brief description of the new CNEP program is enclosed in Table 1.
The organisation structure of senior mentors, near-peer mentors and mentees may be found in Fig 1 below. Each senior mentor advised a group of near-peer mentors, who then in turn advised a group of mentees. Mentees and near-peer mentors may give feedback privately to the senior mentor, and mentees may also give feedback privately to their near-peer mentors.
Evaluating the CNEP program
Given the unproven benefits of CNEP mentoring, an evaluation of CNEP mentoring was called for. The research team consisting of a methodologist, an educationalist, a senior librarian, 2 senior clinicians, 4 freshly recruited medical students with no research experience and 4 junior doctors and 4 medical students with previous research experience in the PMI novice mentoring program discussed the issues pertaining to assessing CNEP mentoring with educationalists, clinicians, academics, mentors and administrators at Duke-NUS Medical School, Yong Loo Lin School of Medicine at the National University of Singapore, the University of Liverpool, the National Cancer Centre Singapore and Singapore General Hospital (henceforth the expert team) to delineate the research questions. The primary research question was “what were the mentee’s experiences of the CNEP mentoring program?” and the secondary question was “what was the impact of the addition of e-mentoring and near peer mentoring to the PMI novice mentoring program?”.
However, there were significant limitations in addressing these research questions in the absence of effective assessment tools to evaluate mentee experiences in CNEP mentoring. To overcome this issue the expert and research teams concluded that a new tool to assess mentoring was required.
Methods
A three phased approach was proposed to design a new tool and to address the primary and secondary research questions. Phase 1 involved design of a new tool, Phase 2 involved assessing the mentee’s experiences using the newly designed CNEP mentoring tool and Phase 3 involved comparing the findings of the new tool with data from the PMI novice mentoring program. This process is outlined in Fig 2 below.
Phase 1
A Delphi study [82] was proposed to design a content-valid instrument to obtain mentees’ perceptions of their mentoring experiences and outcomes in the CNEP program.
The first step of the Delphi study sought to provide a set of survey questions that would be used to form the items of the Delphi survey questionnaire [82]. To do so the research team reviewed nine systematic reviews on novice, near-peer, peer and e-mentoring programs [1, 3–6, 12, 13, 19, 20]. In addition, the research team also carried out a systematic scoping review of mentoring tools to extract specific activities identified as being essential in mentorship programs (this will be discussed elsewhere). The findings of the review of these reviews revealed that 5 essential activities in mentoring programs which were 1) matching practices, 2) mentoring relationships, 3) the mentoring environment, 4) the mentoring structure and 5) the mentoring culture.
These 5 essential activities were characterised by a total of 168 activities that were reviewed for redundancies, trimming the list to 67 items [83]. This list of 67 items was then submitted to a purposive sample of 9 mentorship “experts” who have published on mentoring in medical education or who are acknowledged experts in medical education. These experts who were not part of the expert team then rated each of the 67 items for their importance to include in a survey related to measuring the five domains. Experts were asked to rate the item in one of the following four ways: “do not include, present to mentors only, present to mentee only or present to both mentor and mentee”.
A 70% consensus agreement for each item’s inclusion was set a-priori by the research team. Items with more than 70% of participants agreeing that a question should be addressed to either the mentor only, mentee only, or mentor and mentee were eligible for inclusion into the final survey instrument. After the first round of ratings, the number of items was reduced to 33. The items were then formatted into a final survey instrument and sent back to the experts for their assessment of how well each set of items reflected the domain of interest. The final survey received unanimous support from all 7 experts who participated in the second round.
The second stage in the Delphi study involved cognitive interviews with 3 PMI mentees. Cognitive Interviews served to assess the ‘response process validity’, or how prospective participants interpret items within the pilot tool [84] and whether the participant’s interpretation of the questions matches what the survey designer had in mind [85, 86]. These Cognitive Interviews were carried out using a think-aloud approach that was audiotaped and reviewed by a trained interviewer who was also an independent clinician with whom the participants have not worked with before. The ‘think-aloud’ technique required participants to explicitly ‘think aloud’ as they answered the questionnaire to help elucidate how a response was arrived at. The ‘think-aloud’ technique was supplemented with ‘verbal probing’, where questions were asked to ‘probe’ or delve deeper into the basis for a response. This process allowed for open-ended and unanticipated information to be gathered. Minor modification in the wording of the survey instrument were made following the first round of the Cognitive Interviews. In the second round of Cognitive Interviews 3 other mentees were asked to adopt the ‘think-aloud’ technique as they completed the modified survey. Analysis of their experiences revealed no new issues compared to those highlighted in the first round. As a result no substantive changes were made to the survey instrument following the second round.
Phase 2
In Phase 2 the new tool was piloted amongst mentees in the CNEP program. Email invites containing information about the study, the tool and an attached informed consent form was sent to 28 mentees who had completed the CNEP program and had published articles in peer reviewed journals and or had presented posters in conferences between 2016 and 2018. Invitees who agreed to participate in the survey printed out, signed, and scanned or photographed the consent form and returned it to a study team member with whom they have not worked with previously and who is not part of the team analysing the data. Upon receipt of the signed consent form, the study team member then sent a link to the survey that was hosted on a university website.
This study was approved by the Central Institutional Review Board of the National Cancer Centre Singapore (CIRB Ref 2018/2904: Creating a new tool to evaluate Combined novice, peer, near-peer and e-mentoring (C-NEP) mentoring).
Data collection and analysis.
Overall, 18 of 28 of those invited to participate completed the study (10 male and 8 female medical students and postgraduate medical doctors, who began their participations in the PMI at varying years of study in the 5-year undergraduate medical degree at National University of Singapore and Nanyang Technological University). The profile of mentees who participated in the study can be found in Table 2 found in the results section.
Analysis of the qualitative open-ended survey responses.
Three reviewers (YPT, KTT, LK), experienced in the use of Braun and Clarke [87]’s approach to thematic analysis carried out independent analysis of the anonymized qualitative data collected from the open-ended questions. To enhance rigour, analysis of the qualitative data focused upon descriptions of PMI experiences to preserve the participant’s ‘voice’ and the integrity of their ideas, emotions, and beliefs. An inductive approach allowed themes to be “inductively defined from the raw data without any predetermined classification” [88].
In keeping with the first phase of Braun and Clarke [87]’s approach, an iterative step-by-step thematic analysis was carried out. By immersing themselves in the data and making notes, the reviewers sought to find meaning and purpose in the data.
Next, the reviewers constructed ‘codes’ or a “feature of the data (semantic content or latent) that appears interesting to the analyst, and refer to ‘the most basic segment, or element, of the raw data or information that can be assessed in a meaningful way regarding the phenomenon’ [89] from the ‘surface’ meaning of the mentee’s responses. The initial codes from ‘open coding’ were then grouped into categories according to their similarities.
In the third phase of Braun and Clarke [87]’s approach categories were organised into themes that best depict the data. Three reviewers (YPT, KTT, and LK) used mind maps to illustrate the links between the various codes and to help delineate themes whilst the other reviewer employed lists (RK) to identify potential themes which “represents some level of patterned response or meaning within the data set”.
In the fourth phase of Braun and Clarke [87]’s approach each reviewer reviewed and refined their themes to ensure they were coherent and representative of the whole data set.
In this fifth phase of Braun and Clarke [87]’s approach, the reviewers continued to work independently naming and delineating the specific characteristics of each theme. The three reviewers used Sambunjak et al. [90]’s “negotiated consensual validation” approach to agree upon a common coding framework and code book. The code book consisted of the codes, sub-themes, definitions, descriptions of terms and guidelines on when to use and when not to apply [91].
The reviewers independently extracted and classified all quotations using the code books, collapsed them into themes and subthemes [92] and maintained an iterative approach to the analysis [93].
An external reviewer analysed each code for consistency and accuracy [92]. The themes identified by each reviewer were discussed online and at an author’s meeting where consensus on a final list of themes and subthemes was achieved using the “negotiated consensual validation” approach.
Quantitative analysis was limited given the small numbers of respondents.
Triangulation of qualitative and quantitative survey data.
Descriptive analyses were performed on the quantitative survey data, reporting the proportion of Likert scale responses for the respective survey questions. The reviewers compared the quantitative results with the qualitative survey data to propose possible explanations for the quantitative survey responses, and highlight any contradicting data points between the qualitative and quantitative datasets [94].
Validity and reliability of the analysis.
For the purposes of triangulation of coding and thematic analysis of qualitative data, the analysis was carried out by three independent reviewers. The codes and themes identified by each reviewer was discussed by the three reviewers in online and face-to-face meetings. In addition, the findings of each reviewer and their combined analysis was reviewed by an experienced external reviewer well versed in the topic at hand. To further ensure theoretical validation, the results of the analysis was compared with prevailing data. An iterative process was employed which meant that any new codes identified meant that all the survey data were reviewed to verify the classification and ensure complete data extraction.
Results
Demographic data
Thematic analysis of the responses to the open-ended questions in the survey revealed 3 themes including CNEP mentoring process, the benefits and the challenges of CNEP mentoring. The themes expand upon the descriptive statistical analysis of specific close-ended and Likert scale responses of the survey, which will be discussed in tandem.
A. CNEP mentoring process
There are three sub-themes delineated–“mentoring stages”, “e-mentoring’s role” and the “near-peer mentoring’s role”.
1) Mentoring stages.
There are 6 subthemes to mentoring stages. These include recruitment, aligning expectations, mentoring process, mentoring relationships, mentoring environment and near peer role.
i. Recruitment. Eighty-nine percent (89%, n = 16) of mentees selected their own senior mentor whilst only 33% (n = 6) of mentees selected their near-peer mentor. Mentor selection was based upon a variety of factors. Prior interactions with the mentor and recommendations from colleagues were important considerations as were the mentor’s personal attributes, experience and common personal and professional interests (Table 3).
In most cases the near-peer mentor was often determined by the senior mentor (n = 12). The role of the senior mentor was determined by research, clinical and educational experiences (n = 18, Table 4).
In 89% (n = 16) of case mentees saw the senior mentor as the consultant/attending. The near-peer mentor was in most cases a senior resident or resident (n = 12, Table 5).
ii. Aligning expectations (n = 11). Once mentees and mentors agreed to work together, they met to align expectations (n = 18), availability (n = 18), goals (n = 15) and timelines (n = 12) and establish their roles and responsibility (n = 11, Table 6). Mentees saw this process as critical (n = 3) and as a means to “clarify whatever doubt and expectations about the process beforehand” (Mentee 17).
iii. Mentoring process. The PMI CNEP mentoring program is built around a structured research process. This research-based approach guided the mentoring approach and ensured that it is easily understood by mentees, near-peer mentors and senior mentors as well as the host organization and applied in a consistent fashion. The regular steps involved in the research process including the design, data gathering, review of the study findings, analysis of the data, manuscript preparation, and ending with reflection of the research process ensured a systematic mentoring trajectory replete with clear stages that can be evaluated longitudinally.
iv. Evolving mentoring relationship (n = 17). A key feature in the mentoring process was change. This was primarily the result of changing schedules and priorities, access to training and support and the inevitable difference in the focus of each stage of the research based mentoring process. The personal and professional needs of the mentee also changed the mentoring relationship (Table 7).
As a result, there were regular reviews of milestones and discussions to resolve issues, realign goals and expectations, and provide specific trainings and resources to empower mentees in the research process.
- a) Review milestones (n = 2)
“It (mentee role) did (change), based on changes and needs of the relationship.”
(Mentee 1)
- b) Resolve issues (n = 3)
“I am thankful that they are always there to patiently communicate ideas with me, remain open and honest in all interactions, and willing to clarify any misunderstandings so that my relationships with them continue to be built on trust.”
(Mentee 15)
- c) Realign goals and expectations (n = 1)
“Set good deadlines and negotiate where needed.”
(Mentee 4)
- d) Provide specific training and resources (n = 1)
“to be proactive and communicate well with mentors on goals, responsibilities and expectations”
(Mentee 18)
v. Mentoring environment (n = 17). Change within the mentoring relationship however was possible given the impact of the mentoring environment and the mentee’s own attitudes and motivations (Table 8).
The mentoring environment also helped support the mentees during trying times particularly in the face of competing commitments (n = 16), psychological and emotional issues (n = 9) and in addressing their professional identities along the course of the mentoring process (n = 9, Table 9).
vi. Transitioning to a near-peer mentor role (n = 3). As mentees progressed, their mentors evaluated the development of their mentees and their suitability as near-peer mentors.
“Yes, I slowly changed from the role of a mentee to near-peer mentor under their guidance. My (senior) mentor's role stayed the same, but with the change in my role as near-peer mentor, my (near-peer) mentor's role also changed to become senior near-peer mentor, whereby my (near-peer) mentor will mentor me to become a better near-peer mentor.”
(Mentee 15)
One of the motivations to take up the role of a near-peer mentor is to pay it forward.
“I became a near-peer mentor for other juniors too- paying it forward”
(Mentee 3)
As the mentees reflected upon their mentoring experiences and growth, the majority of the mentees had a positive mentoring experience, with 94% (n = 17) of them feeling a sense of belonging, teamwork and collegiality.
2) Electronic-mentoring’s role (n = 11).
The primary role of electronic mentoring was to supplement face-to-face meetings (n = 10). Mentees felt face-to-face helped build connections and appreciated the nonverbal communication it provided. As a result, mentees believed electronic mentoring could not replace face-to-face meetings (n = 10).
“Convenient but somewhat more impersonal; meant to complement other aspects of mentoring”
(Mentee 9)
Electronic-mentoring however did enhance access to support and feedback (n = 11).
“Faster/ more efficient communication and dissemination of information”
(Mentee 9)
The asynchronous nature of Electronic-mentoring also facilitated reflective practice (n = 1).
“We do not necessary have to reply immediately and can always take time to reflect prior to replying”
(Mentee 15)
Electronic-mentoring was also seen as an important platform for tracking and disseminating information (n = 1).
“Have details always accessible for better follow up and work.”
(Mentee 4)
3) Near-peer mentor’s role.
89% (n = 16) of the mentees felt that the near-peer mentor played a pivotal role in the development of personalized and nurturing mentoring relationship. Effective near-peer mentors were committed, empathetic, non-judgemental, supportive, caring approachable and accessible (n = 17).
“A more junior mentor may be more relatable/approachable to a medical student”
(Mentee 3)
“Perceived as less authoritative/threatening, more collegial environment as responses are more polite, especially helpful climate for novices who may be more shy.”
(Mentee 4)
Near-peers facilitated the timely and personalized support (n = 12).
“One can rely on the secondary mentor should the (senior) mentor be unavailable … multiple mentors may provide a more holistic experience to mentees.”
(Mentee 13)
Near peer mentors provided an alternate source of support (n = 9).
“Spent more time understanding mentee's concerns and sharing valuable experiences and insight”
(Mentee 3)
The near-peer mentors also provided oversight (n = 3).
“(Having a near-peer mentor) definitely helps to keep track of work more closely”
(Mentee 16)
B. Benefits of CNEP mentoring
Mentees listed the benefits of the program in Table 10.
C. Challenges of CNEP mentoring
There are a number of specific challenges to use of the PMI’s CNEP mentoring approach. These are:
The presence of multiple avenues of communication may become a source of misunderstanding (n = 1).
“Communication and clarification regarding more complex/confusing matters is easier when done face-to-face compared to electronic-mentoring (only)”
(Mentee 13)
Near peers may also be the source of misunderstandings (n = 1).
“(near-peer) and (senior) mentors may have differing views/instructions/perspectives which may confuse medical students”
(Mentee 13)
Use of CNEP mentoring blurred professional and personal boundaries (n = 1).
“Invasion into private time/after hours, but this is universal of communication technologies”
(Mentee 4)
This contributed to stress amongst 67% (n = 12) of mentees and a negative impact upon their work life balance amongst 28% of mentees (n = 5, Table 11).
An expansive program also raises concerns about ethical practice in the mentoring program (Table 12).
Details of issues faced such as ethical issues, unfair treatment and mentoring malpractice were not expanded upon.
Phase 3
Based upon advice from the expert team and stakeholders as part of the sixth phase of Braun and Clarke [87]’s approach, comparisons of mentee’s experiences in the CNEP program and accounts of mentoring experiences PMI mentoring set out in Krishna et al. [21]’s study were carried out.
Whilst rather superficial given that these comparisons involve qualitative data it is nonetheless reassuring that the themes and accounts detailed in both studies were similar. In addition it lends weight to the validity of the tool designed here given that Krishna et al. [21]’s study employed semi-structured interviews that are seen as the ‘gold standard’ for assessing mentoring experiences.
In Krishna et al. [21]’s study, the novice mentoring process was seen to evolve in competency-based stages that were described as ‘circumscribed sequential projects’ with ‘specific goals and competency requirements’ that build on one another to achieve an overarching goal. These competency-based stages pivoted on effective communications between mentees and mentors, the establishment of clear expectations, codes of conduct and timelines and the alignment of mentoring goals. The stages of the PMI novice mentoring process were based upon the research process and developed along the course of the research process. These stages include goals setting, data gathering, manuscript writing, submission and responding to the journal’s queries. With PMI CNEP mentoring similarly focused, it is anticipated that these stages are expected to be similar and variations in practice ought to reflect the impact of NP and e-mentoring. Closer scrutiny of these findings will follow.
Discussion
To address its primary research question, this study has adopted a three phased research approach to circumnavigate the lack of effective assessments tools to evaluate CNEP mentoring. Phase 1 identified and extracted specific activities deemed essential in mentorship programs to evaluate the effects of NP and e-mentoring. Through the Delphi process, a new tool was created and piloted through use of Cognitive Interviews.
Phase 2 saw application of the new tool to capture a mentee’s perspective of the PMI’s CNEP mentoring program. This data is consistent with prevailing accounts of mentoring experiences in novice mentoring programs in Palliative Medicine [1, 4, 10, 12, 13, 20, 21].
In Phase 3 data from the new tool on mentee’s experiences in CNEP experiences were compared with mentee accounts of their experiences in the PMI’s novice mentoring program. These comparisons revealed that CNEP mentoring like the PMI novice mentoring process progressed through the mentoring stages of recruitment, alignment of the mentoring process, data collection, data analysis and preparation for publication. Here similarities between PMI’s previous and present mentoring approaches reaffirm the validity of comparing the findings of the two studies to help discern the impact of changes to the novice mentoring program.
The presence of mentoring stages suggests the presence of two aspects of CNEP mentoring. First is that progress from one stage to another is competency-based and second, CNEP mentoring is dependent upon a structured mentoring approach. In turn these findings have two important implications. A competency-based approach ensures that mentees only progress upon attainment of required skills, reiterating the notion of each stage is a ‘circumscribed sequential projects’ with ‘specific goals and competency requirements’. It also ensures consistent mentoring experiences and clear end points that help alignment of expectations, goals, timelines, expectations and roles and responsibilities [96]. Consistency in the mentoring approach is enhanced by the presence of codes of conduct and the stages of mentoring. The importance of consistency within the mentoring process is underlined by ‘confusion’ amongst mentees when senior and near-peer mentors are not in sync with their advice.
However, it is also clear that the framework can ill afford to be rigid given the need to contend with personalization of the mentoring process. Efforts to personalise the mentoring process was evident in catering meeting and training schedules to the mentee’s abilities, availability and goals. Personalization of the mentoring process was also evident in determining the timelines, codes of conduct, roles and responsibilities of each stakeholder. Ensuring effective balance between a consistent approach and flexibility are personalised, appropriate, specific, timely, longitudinal, accessible and holistic assessments of the mentoring process that inform the administrators and mentors of the adaptations needed.
However, the CNEP mentoring program also throws up new considerations. Unlike the PMI’s novice mentoring program that employed mentee-initiated matching that saw mentee’s select their own mentors, CNEP mentoring employed a combination of mentee-initiated mentor matching and matching of near-peer mentors and mentees. Selection of the near-peer mentor was determined by the senior mentor and was often determined by the research topic of research. There is little evidence of any ill effects to the overall success of mentoring relationships. This may be a reflection of the undiminished primacy of the senior mentor’s role with near-peer mentors providing only practical and psychosocial support, that all near-peer mentors are experienced clinically and from a research perspective, trained and employ the same mentoring approach meaning that there is consistency within the overall mentoring process, the belief that all mentoring interactions are carried out under the oversight of the mentor by virtue of the e-mentoring platform or the fact that e-mentoring allows immediate and easy means of contact with the senior mentor. Many of these possible explanations also raise questions as to whether having NP mentoring and e-mentoring addresses the shortage of mentoring support or merely adds to the senior mentor’s workload. Yet with no failed relationships reported, it would seem that CNEP mentoring appears to be as effective as the PMI’s novice mentoring program.
Once more communications play a critical role in the mentoring process, helping to align expectations, provide feedback and resolve issues. Here it would seem having both the presence of near-peer mentors and e-mentoring support enhanced communication and cultivated a more nurturing mentoring environment and relationship that enhanced the mentoring experience. The combination of NP and e-mentoring and novice mentoring is also seen as a complementary process particularly with data suggesting that neither NP mentoring or e-mentoring was seen as ‘stand-alone’ mentoring approaches. E-mentoring was ‘found to be ‘impersonal’ (Mentee 9) and did not necessarily result in a timely response (Mentee 15) whilst NP mentoring sometimes led to misunderstandings (Mentee 13) and sometimes invasions of private time (Mentee 4). It would seem novice mentoring and oversight and support by the senior mentor was crucial to the overall mentoring process.
However not all the shortfalls facing NP and e-mentoring can be addressed by supplementing them with novice mentoring. The data highlights a number of ethical issues. Perhaps none are so concerning as the worries that 11% of mentees believed that there were ethical issues concerning CNEP mentoring including mentees believing that their near-peer and or senior mentor was motivated by self-interests and/or treated them unfairly. This raises questions as to why 10 of the invited mentees did not participate in the survey with 67% (n = 12) of mentees reported being stressed, 28% (n = 5) experiencing negative effects upon their work life balance and 17% (n = 3) reporting negative impacts upon their clinical work and studies. This is a concern given that CNEP mentoring is lauded for increasing personalised, appropriate, specific, timely, holistic, accessible and longitudinal support and feedback. These findings underline the need for close monitoring of the CNEP program.
The need for effective assessment and oversight of the mentoring process underlines the key role of the host organization. The host organization plays a number of critical roles including recruiting and training mentees and mentors, overseeing and supporting the matching process and structuring the trajectory of the mentoring process around the research process. The other critical role of the host organization lies in its establishment of clear codes of conduct, setting out the roles and responsibilities of mentees, near peer mentors and senior mentors and ensuring effective oversight of the mentoring process. The host organization also play a critical role in supporting mentees particular when some have reported a loss of self-esteem, being treated unfairly and disillusionment. The role, ability and support for the host organization must also be evaluated and must be part of the audit and policing of mentoring processes.
The absence of such holistic assessments re-emphasise the need to move away from reliance upon post-mentoring surveys and snap-shot interviews. It is clear that views and concerns of senior and near-peer mentors and representatives of the host organization are required given that data on their interactions will change the complexion of the discussion and better inform practice and policing of the mentoring program. To enable such a process a mentoring portfolio replete with both mentoring diaries that would capture all interactions between mentee and the mentors and regular input from the mentors on the mentee’s progress and general direction of the progress may be required. For the host organization, feedback loops must be closed and independent assessments of the mentee, senior and near-peer mentors are required. This can capture concerns on the part of any stakeholder throughout the course of the mentoring process and ensure personalised, appropriate, specific, timely, holistic, accessible and longitudinal support and prompt and appropriate training be provided to each party. Evaluations of the host organization itself and the mentoring program as a whole should be carried out regularly to help understand some of the influences upon individual relationships and provide a more complete picture of the mentoring relationship.
Overall, CNEP mentoring does fill many of the gaps in novice mentoring but this approach can only do so when clear lines of communication, effective codes of practice and appropriate oversight is provided. As a result, like novice mentoring, CNEP mentoring needs careful and oversight by the host organization.
Limitations
Whilst a mixed-method questionnaire approach to appraising mentoring relationships and programs is less resource intensive and offers deeper insights into the CNEP mentoring experiences, it has not yet been validated. Similarly, whilst the themes are similar to those identified in semi-structured interviews of the PMI program, a formal validation program for the new is being planned.
The use of retrospective data at a single time point a variable distance from the end of the mentoring process invites recall bias [97]. The presence of single data points also limits the depth to which it is possible to delve into the various aspects of the mentoring process and the mentoring experiences.
The small sample size and a uniquely structured approach around a research process may limit the applicability of these findings in other settings.
Conclusion
It is apparent that CNEP mentoring offers a new dimension to mentoring but to realise its full potential CNEP mentoring requires clear mentoring guidelines, mentor, near peer mentor and mentee training, effective holistic and longitudinal assessment of mentoring processes and support [25, 26, 28–32, 35–40, 67, 98–111]. Addressing these issues must be a priority for CNEP program designers and administrators. Fulfilling CNEP mentoring potential must be informed by closer scrutiny of the accounts of ethical issues in mentoring highlighted in recent reviews [22] and supplemented by new systematic reviews of training and coordination of mentoring efforts and mentoring dynamics between mentors and near peer mentors. It is only with these insights and effective support from host organizations can CNEP mentoring play its role in medical education.
Acknowledgments
The authors would like to dedicate this paper to the late Dr S Radha Krishna whose advice and ideas were integral to the success of this study. The authors would like to thank the anonymous reviewers whose advice and feedback greatly improved this manuscript.
References
- 1. Toh YP, Lam BL, Soo J, Chua KLL, Krishna L. Developing Palliative Care Physicians through Mentoring Relationships. Palliat Med Care 2017;4(1).
- 2. Yap HW, Chua J, Toh YP, Choi HJ, Mattar S, Kanesvaran R, et al. Thematic Review of Mentoring in Occupational Therapy and Physiotherapy between 2000 and 2015, Sitting Occupational Therapy and Physiotherapy in A Holistic Palliative Medicine Multidisciplinary Mentoring Program. Journal of Palliative care and Pediatrics. 2017;2(1):46–55.
- 3. Sng JH, Pei Y, Toh YP, Peh TY, Neo SH, Krishna LKR. Mentoring relationships between senior physicians and junior doctors and/or medical students: A thematic review. Med Teach. 2017;39(8):866–75. pmid:28562193
- 4. Ikbal MFBM, Wu JT, Wahab MT, Kanesvaran R, Krishna LKR. Mentoring in Palliative Medicine: Guiding Program Design through Thematic Analysis of Mentoring in Internal Medicine between 2000 and 2015. J Palliat Care Med 2017;7(5).
- 5. Tan B, Toh YL, Toh YP, Kanesvaran R, Krishna LKR. Extending Mentoring in Palliative Medicine-Systematic Review on Peer, Near-Peer and Group Mentoring in General Medicine. Journal of Palliative Care & Medicine. 2017;07(06).
- 6. Tan YS, Teo SWA, Pei Y, Sng JH, Yap HW, Toh YP, et al. A framework for mentoring of medical students: thematic analysis of mentoring programmes between 2000 and 2015. Advances in Health Sciences Education. 2018:1–27.
- 7. Sheri K, Too JYJ, Chuah SEL, Toh YP, Mason S, Krishna LKR. A scoping review of mentor training programs in medicine between 1990 and 2017. Medical Education Online. 2019;24(1):1555435. pmid:31671284
- 8. Chang MK, Lim MX, Tay K, Lee RJ, Sim SW, Menon S, et al. The Influence of Evolving Confucian Beliefs in the Selection of Proxy Decision-Makers at the End of Life in Singapore. Asian Bioethics Review. 2017.
- 9. Low CQT, Toh YL, Teo SWA, Toh YP, Krishna L. A narrative review of mentoring programmes in general practice. Education for Primary Care. 2018;29(5):259–67. pmid:30059278
- 10. Toh YP, Karthik R, Teo CC, Suppiah S, Cheung SL, Krishna L. Toward Mentoring in Palliative Social Work: A Narrative Review of Mentoring Programs in Social Work. American Journal of Hospice and Palliative Medicine®. 2017;35(3):523–31.
- 11. Lin J, Chew YR, Toh YP, Krishna LKR. Mentoring in nursing: an integrative review of commentaries, editorials, and perspectives papers. Nurse educator. 2018;43(1):E1–E5. pmid:28492413
- 12. Wahab MT, Ikbal MFBM, Wu J, Loo WTW, Kanesvaran R, Krishna LKR. Creating Effective Interprofessional Mentoring Relationships in Palliative Care- Lessons from Medicine, Nursing, Surgery and Social Work. Journal of Palliative Care & Medicine. 2016;06(06).
- 13. Wu J, Wahab MT, Ikbal MFBM, Loo TWW, Kanesvaran R, Krishna LKR. Toward an Interprofessional Mentoring Program in Palliative Care—A Review of Undergraduate and Postgraduate Mentoring in Medicine, Nursing, Surgery and Social Work. Journal of Palliative Care & Medicine. 2016;06(06):1–11.
- 14. Carey EC, Weissman DE. Understanding and finding mentorship: A review for junior faculty. J Palliative Med. 2010;13(11):1373–9.
- 15. Jackson V, Arnold RM. A model of mosaic mentoring. J Palliat Med. 2010;13(11):1371. pmid:21091021
- 16. Arnold RM. Mentoring the next generation: a critical task for palliative medicine. J Palliative Med. 2005;8(4):696–8.
- 17. Case AA, Orrange SM, Weissman DE. Palliative medicine physician education in the United States: a historical review. J Palliative Med. 2013;16(3):230–6.
- 18. Periyakoil VS. Declaration of interdependence: The need for mosaic mentoring in palliative care. J Palliative Med. 2007;10(5):1048–9.
- 19. Hee JM, Yap HW, Ong ZX, Quek SQM, Toh YP, Mason S, et al. Understanding the Mentoring Environment Through Thematic Analysis of the Learning Environment in Medical Education: a Systematic Review. Journal of general internal medicine. 2019:1–10.
- 20. Loo WTW, Ikbal MFBM, Wu JT, Wahab MT, Yeam CT, Ee HFM, et al. Towards a Practice Guided Evidence Based Theory of Mentoring in Palliative Care. J Palliat Care Med 2017;7(1).
- 21. Krishna L, Toh Y, Mason S, Kanesvaran R. Mentoring stages: A study of undergraduate mentoring in palliative medicine in Singapore. PloS one. 2019;14(4):e0214643–e. pmid:31017941
- 22. Krishna LKR, Renganathan Y, Tay KT, Tan BJX, Chong JY, Ching AH, et al. Educational roles as a continuum of mentoring’s role in medicine–a systematic review and thematic analysis of educational studies from 2000 to 2018. BMC medical education. 2019;19(1):439. pmid:31775732
- 23. Hollinderbäumer A, Hartz T, Ückert F. Education 2.0-How has social media and Web 2.0 been integrated into medical education? A systematical literature review. GMS Zeitschrift für medizinische Ausbildung. 2013;30(1).
- 24. Cartledge P, Miller M, Phillips B. The use of social-networking sites in medical education. Medical Teacher. 2013;35(10):847–57. pmid:23841681
- 25. Gray K, Annabell L, Kennedy G. Medical students’ use of Facebook to support learning: Insights from four case studies. Medical teacher. 2010;32(12):971–6. pmid:21090950
- 26. Nicolai L, Schmidbauer M, Gradel M, Ferch S, Antón S, Hoppe B, et al. Facebook groups as a powerful and dynamic tool in medical education: mixed-method study. Journal of medical Internet research. 2017;19(12):e408. pmid:29273572
- 27. Vogelsang M, Rockenbauch K, Wrigge H, Heinke W, Hempel G. Medical Education for “Generation Z”: Everything online?!–An analysis of Internet-based media use by teachers in medicine. GMS journal for medical education. 2018;35(2).
- 28. Shenouda JE, Davies BS, Haq I. The role of the smartphone in the transition from medical student to foundation trainee: a qualitative interview and focus group study. BMC medical education. 2018;18(1):175. pmid:30064424
- 29. Masters K, Ellaway RH, Topps D, Archibald D, Hogue RJ. Mobile technologies in medical education: AMEE Guide No. 105. Medical teacher. 2016;38(6):537–49. pmid:27010681
- 30. Pander T, Pinilla S, Dimitriadis K, Fischer MR. The use of Facebook in medical education–A literature review. GMS Zeitschrift für Medizinische Ausbildung. 2014;31(3).
- 31. Raiman L, Antbring R, Mahmood A. WhatsApp messenger as a tool to supplement medical education for medical students on clinical attachment. BMC medical education. 2017;17(1):7. pmid:28061777
- 32. Pinilla S, Nicolai L, Gradel M, Pander T, Fischer MR, von der Borch P, et al. Undergraduate medical students using Facebook as a peer-mentoring platform: a mixed-methods study. JMIR medical education. 2015;1(2):e12. pmid:27731859
- 33. Sandars J, Homer M, Pell G, Croker T. Web 2.0 and social software: the medical student way of e-learning. Medical teacher. 2008;30(3):308–12. pmid:18608950
- 34. Kind T, Patel PD, Lie D, Chretien KC. Twelve tips for using social media as a medical educator. Medical teacher. 2014;36(4):284–90. pmid:24261897
- 35. John B. Kinship-King's Social Harmonisation Project. Pilot phase of a social network for use in Higher Education (HE). Bulletin du Groupement International Pour la Recherche Scientifique En Stomatologie et Odontologie. 2012;51(3):19–22.
- 36. Sutherland S, Jalali A. Social media as an open-learning resource in medical education: current perspectives. Advances in medical education and practice. 2017;8:369. pmid:28652840
- 37. Schichtel M. A conceptual description of potential scenarios of e-mentoring in GP specialist training. Education for Primary Care. 2009;20(5):360–4. pmid:19849902
- 38. Griffiths M, Miller H. E-mentoring: Does it have a place in medicine? Postgraduate medical journal. 2005;81(956):389–90. pmid:15937205
- 39. Schichtel M. Core-competence skills in e-mentoring for medical educators: A conceptual exploration. Medical teacher. 2010;32(7):e248–e62. pmid:20653366
- 40. Alamro AS, Schofield S. Supporting traditional PBL with online discussion forums: A study from Qassim Medical School. Medical teacher. 2012;34(sup1):S20–S4.
- 41. Fnais N, Soobiah C, Chen MH, Lillie E, Perrier L, Tashkhandi M, et al. Harassment and Discrimination in Medical Training. Academic Medicine. 2014;89(5):817–27. pmid:24667512
- 42. Babaria P, Abedin S, Berg D, Nunez-Smith M. “I'm too used to it”: A longitudinal qualitative study of third year female medical students' experiences of gendered encounters in medical education. Social Science & Medicine. 2012;74(7):1013–20.
- 43. Jagsi R. Sexual Harassment in Medicine—#MeToo. New England Journal of Medicine. 2018;378(3):209–11. pmid:29236567
- 44.
Prime J, Moss-Racusin C. Engaging men in gender initiatives: what change agents need to know. New York: Catalyst; 2009.
- 45. Johnson W, Smith D. Men shouldn’t refuse to be alone with female colleagues. Harvard Business Review; 2017.
- 46.
Cain MC. Unintended consequences of sexual harassment scandals. New York Times; 2017.
- 47. Freischlag JA, Faria P. It Is Time for Women (and Men) to Be Brave. Jama. 2018;319(17):1761. pmid:29610840
- 48.
Lautenberger D, Dandar V, Raezer C, Sloane R. The state of women in academic medicine: the pipeline and pathways to leadership, 2013–2014. Washington, DC: Association of American Medical Colleges; 2014.
- 49. Byerley JS. Mentoring in the Era of# MeToo. Jama. 2018;319(12):1199–200. pmid:29584847
- 50.
Wilcox C. Scapegoat: targeted for blame. Denver: Outskirts Press; 2010.
- 51. Hubbard P. Fear and loathing at the multiplex: everyday anxiety in the post-industrial city. Capital & Class. 2003;27(2):51–75.
- 52.
Furedi F. The politics of fear: beyond left and right. London: Continuum Press; 2006.
- 53. Carr PL, Gunn CM, Kaplan SA, Raj A, Freund KM. Inadequate Progress for Women in Academic Medicine: Findings from the National Faculty Study. Journal of Women's Health. 2015;24(3):190–9. pmid:25658907
- 54.
Glassner B. The Culture of Fear: Why Americans Are Afraid of the Wrong Things: Crime, Drugs, Minorities, Teen Moms, Killer Kids, Muta: Basic books; 2010.
- 55.
Sexual harassment backlash survey. Lean In; 2018.
- 56. Soklaridis S, Zahn C, Kuper A, Gillis D, Taylor VH, Whitehead C. Men's Fear of Mentoring in the #MeToo Era—What's at Stake for Academic Medicine? N Engl J Med. 2018.
- 57. Long J. The dark side of mentoring. The Australian Educational Researcher. 1997;24(2).
- 58.
Duck S. Stratagems, spoils, and a serpent's tooth: On the delights and dilemmas of personal relationships. In: Cupach WR, Spitzberg BH, editors. The dark side of interpersonal communication. New Jersey, USA: Taylor & Francis; 1994. p. 3–24.
- 59. Chopra V, Edelson DP, Saint S. Mentorship Malpractice. Jama. 2016;315(14):1453–4. pmid:27115263
- 60. Walensky RP, Kim Y, Chang Y, Porneala BC, Bristol MN, Armstrong K, et al. The impact of active mentorship: results from a survey of faculty in the Department of Medicine at Massachusetts General Hospital. BMC medical education. 2018;18(1):108. pmid:29751796
- 61. Singh TSS, Singh A. Abusive culture in medical education: Mentors must mend their ways. J Anaesth Clin Pharm. 2018;34(2):145–7.
- 62. Olasoji HO. Broadening conceptions of medical student mistreatment during clinical teaching: message from a study of “toxic” phenomenon during bedside teaching. Advances in medical education and practice. 2018;9:483. pmid:29950919
- 63. Lee FQH, Chua WJ, Cheong CWS, Tay KT, Hian EKY, Chin AMC, et al. A Systematic Scoping Review of Ethical Issues in Mentoring in Surgery. Journal of Medical Education and Curricular Development. 2019;6:2382120519888915.
- 64. Cheong CWS, Chia EWY, Tay KT, Chua WJ, Lee FQH, Koh EYH, et al. A systematic scoping review of ethical issues in mentoring in internal medicine, family medicine and academic medicine. Advances in Health Sciences Education. 2019:1–25.
- 65. Lim SYS, Koh EYH, Tan BJX, Toh YP, Mason S, Krishna LK. Enhancing geriatric oncology training through a combination of novice mentoring and peer and near-peer mentoring: A thematic analysis of mentoring in medicine between 2000 and 2017. Journal of geriatric oncology. 2019.
- 66. Chong JY, Ching AH, Renganathan Y, Lim WQ, Toh YP, Mason S, et al. Enhancing mentoring experiences through e-mentoring: a systematic scoping review of e-mentoring programs between 2000 and 2017. Advances in Health Sciences Education. 2019:1–32.
- 67. Walsh K. Online mentoring in medical education. South African Family Practice. 2015:1–2.
- 68. Olaussen A, Reddy P, Irvine S, Williams B. Peer-assisted learning: time for nomenclature clarification. Medical education online. 2016;21(1):30974.
- 69. Stenfors-Hayes T, Kalen S, Hult H, Dahlgren LO, Hindbeck H, Ponzer S. Being a mentor for undergraduate medical students enhances personal and professional development. Med Teach. 2010;32(2):148–53. pmid:20163231
- 70. Boninger M, Troen P, Green E, Borkan J, Lance-Jones C, Humphrey A, et al. Implementation of a longitudinal mentored scholarly project: an approach at two medical schools. Acad Med. 2010;85(3):429–37. pmid:20182115
- 71. Kalen S, Stenfors-Hayes T, Hylin U, Larm MF, Hindbeck H, Ponzer S. Mentoring medical students during clinical courses: a way to enhance professional development. Med Teach. 2010;32(8):e315–21. pmid:20662566
- 72. Watson JC, Clement D., Blom L.C., Grindley E. Mentoring: Processes and Perceptions of Sport and Exercise Psychology Graduate Students. J Appl Sport Psychol 2009;21:231–46.
- 73. von der Borch P, Dimitriadis K, Stormann S, Meinel FG, Moder S, Reincke M, et al. A Novel Large-scale Mentoring Program for Medical Students based on a Quantitative and Qualitative Needs Analysis. GMS Z Med Ausbild. 2011;28(2):Doc26. pmid:21818236
- 74. Clark RA, Sherry LH, Johnson WB. Mentoring relationships in Clinical Psychology Doctoral Training: Results of a National Survey. Teach Psychol. 2000;27(4):262–8.
- 75. Indyk D, Deen D, Fornari A, Santos MT, Lu WH, Rucker L. The influence of longitudinal mentoring on medical student selection of primary care residencies. BMC medical education. 2011;11:27. pmid:21635770
- 76. Gotterer GS, O'day D, Miller BM. The Emphasis program: a scholarly concentrations program at Vanderbilt University School of Medicine. Academic Medicine. 2010;85(11):1717–24. pmid:20671539
- 77. Mark S, Link H, Morahan PS, Pololi L, Reznik V, Tropez-Sims S. Innovative mentoring programs to promote gender equity in academic medicine. Academic Medicine. 2001;76(1):39–42. pmid:11154192
- 78. Pololi LH, Evans AT, Civian JT, Vasiliou V, Coplit LD, Gillum LH, et al. Mentoring Faculty: A US National Survey of Its Adequacy and Linkage to Culture in Academic Health Centers. J Contin Educ Health Prof. 2015;35(3):176–84. pmid:26378423
- 79. Aagaard EM, Hauer KE. A cross-sectional descriptive study of mentoring relationships formed by medical students. J Gen Intern Med. 2003;18(4):298–302. pmid:12709098
- 80. Usmani A, Omaeer Q, Sultan ST. Mentoring undergraduate medical students: experience from Bahria University Karachi. JPMA-Journal of the Pakistan Medical Association. 2011;61(8):790.
- 81. Oelschlager AM, Smith S, Tamura G, Carline J, Dobie S. Where do medical students turn? The role of the assigned mentor in the fabric of support during medical school. Teach Learn Med. 2011;23(2):112–7. pmid:21516596
- 82. Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and Nominal Group in medical education research. Medical teacher. 2017;39(1):14–9. pmid:27841062
- 83. Magee C, Rickards G., Byars LA, Artino AR Jr. Tracing the steps of survey design: a graduate medical education research example. Journal of Graduate Medical Education. 2013;5(1):1–5. pmid:24404217
- 84.
AERA A. NCME. Standards for Educational and Psychological Testing American Educational Research Association 2nd ed Washington DC. 1999.
- 85.
Willis G, Lawrence D, Thompson F, Kudela M, Levin K, Miller K, editors. The use of cognitive interviewing to evaluate translated survey questions: lessons learned. Conference of the Federal Committee on Statistical Methodology, Arlington, VA; 2005: Citeseer.
- 86. Karabenick SA, Woolley ME, Friedel JM, Ammon BV, Blazevski J, Bonney CR, et al. Cognitive processing of self-report items in educational research: Do they think what we mean? Educational Psychologist. 2007;42(3):139–51.
- 87. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology 2006;3(2):77–101.
- 88. Cassol H, Pétré B, Degrange S, Martial C, Charland-Verville V, Lallier F, et al. Qualitative thematic analysis of the phenomenology of near-death experiences. PloS one. 2018;13(2):e0193001. pmid:29444184
- 89.
Boyatzis RE. Transforming qualitative information: Thematic analysis and code development: sage; 1998.
- 90. Sambunjak D, Straus SE, Marusic A. A systematic review of qualitative research on the meaning and characteristics of mentoring in academic medicine. J Gen Intern Med. 2010;25(1):72–8. pmid:19924490
- 91. Chesang K, Hornston S, Muhenje O, Saliku T, Mirjahangir J, Viitanen A, et al. Healthcare provider perspectives on managing sexually transmitted infections in HIV care settings in Kenya: A qualitative thematic analysis. PLoS medicine. 2017;14(12):e1002480. pmid:29281636
- 92. Roze des Ordons AL, Lockyer J, Hartwick M, Sarti A, Ajjawi R. An exploration of contextual dimensions impacting goals of care conversations in postgraduate medical education. BMC Palliat Care. 2016;15:34. pmid:27001665
- 93. Price S, Schofield S. How do junior doctors in the UK learn to provide end of life care: a qualitative evaluation of postgraduate education. BMC Palliat Care. 2015;14:45. pmid:26399845
- 94. Lingard L, Albert M, Levinson W. Grounded theory, mixed methods, and action research. BMJ. 2008;337:a567. pmid:18687728
- 95. O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Academic Medicine. 2014;89(9):1245–51. pmid:24979285
- 96. Lurie SJ. History and practice of competency‐based assessment. Medical education. 2012;46(1):49–57. pmid:22150196
- 97. Mennin S. Self‐organisation, integration and curriculum in the complex world of medical education. Medical education. 2010;44(1):20–30. pmid:20078753
- 98. Kamin C, Glicken A, Hall M, Quarantillo B, Merenstein G. Evaluation of electronic discussion groups as a teaching/learning strategy in an evidence-based medicine course: A pilot study. EDUCATION FOR HEALTH-ABINGDON-CARFAX PUBLISHING LIMITED-. 2001;14(1):21–32.
- 99. Johnston MJ, King D, Arora S, Behar N, Athanasiou T, Sevdalis N, et al. Smartphones let surgeons know WhatsApp: an analysis of communication in emergency surgical teams. The American Journal of Surgery. 2015;209(1):45–51. pmid:25454952
- 100. Hossain IT, Mughal U, Atalla B, Franka M, Siddiqui S, Muntasir M. Instant messaging–one solution to doctor–student communication? Medical education online. 2015;20.
- 101. Hartnup B, Dong L, Eisingerich AB. How an environment of stress and social risk shapes student engagement with social media as potential digital learning platforms: qualitative study. JMIR medical education. 2018;4(2):e10069. pmid:30006324
- 102. Nadir R, Bashir K, Nasir MA, Khan HA. Social Networking Sites (SNSs) in medical education: a student's perspective. Medical education online. 2018;23(1):1524689-. pmid:30252622
- 103.
Saparova D, Williams JA, Inabnit CK, Fiesta M, editors. Information behavior shift: How and why medical students use Facebook. Proceedings of the 76th ASIS&T Annual Meeting: Beyond the Cloud: Rethinking Information Boundaries; 2013: American Society for Information Science.
- 104. Forgie SE, Duff JP, Ross S. Twelve tips for using Twitter as a learning tool in medical education. Medical teacher. 2013;35(1):8–14. pmid:23259608
- 105. Jaffer U, Vaughan-Huxley E, Standfield N, John NW. Medical mentoring via the evolving world wide web. Journal of surgical education. 2013;70(1):121–8. pmid:23337681
- 106. Masters K, Ellaway R. e-Learning in medical education Guide 32 Part 2: Technology, management and design. Medical teacher. 2008;30(5):474–89. pmid:18576186
- 107. Rowe M, Frantz J, Bozalek V. The role of blended learning in the clinical education of healthcare students: a systematic review. Medical teacher. 2012;34(4):e216–e21. pmid:22455712
- 108. Patel PD, Roberts JL, Miller KH, Ziegler C, Ostapchuk M. The responsible use of online social networking: who should mentor medical students. Teaching and learning in medicine. 2012;24(4):348–54. pmid:23036003
- 109. Ellaway R, Masters K. AMEE Guide 32: e-Learning in medical education Part 1: Learning, teaching and assessment. Medical teacher. 2008;30(5):455–73. pmid:18576185
- 110.
Singleton MH. Evaluation of a College of Medicine Peer-Mentoring Program. 2016.
- 111. Martin P, Kumar S, Lizarondo L. Effective use of technology in clinical supervision. Internet interventions. 2017;8:35–9. pmid:30135826