https://orcid.org/0000-0002-8944-4948
Figures
Abstract
Background
Embedding into undergraduate medical programs experiential research curricula, based on holistic theories of education which emphasize participation in the social world, remains uncommon. The purpose of this study was to investigate the journey of undergraduate medical students in relation to an innovative compulsory curriculum-based research module, which has a prominent experiential learning component.
Methods
A convergent mixed methods study design was adapted to develop a systemic understanding of the experience of the undergraduate medical students throughout the respective research module. As such, the students’ perception of the experience was qualitatively explored using thematic analysis (n = 15). In parallel, the students’ performance data were quantitatively analyzed using multi-repeated ANOVA (n = 158). The findings from both types of analyses (i.e., qualitative and quantitative study components) were then mapped onto each using joint display analysis.
Findings
The exploration generated four themes that correspond to sequential steps that the students go through to effectively integrate the scientific research method. These themes include: 1- Attend-Acquire, 2- Accumulate-Assimilate, 3- Apply-Appreciate, and 4-Articulate-Affect. Quantitatively, two distinct clusters of mean Grade Point Average were revealed (p<0.01). Joint display analysis enabled integrating the qualitative and quantitative findings, generating the 8A-Model.
Conclusion
The evidence-driven 8A-Model, generated by this study, highlights that medical students’ understanding of the true value of research seems to increase as they progress in the module. They begin expressing appreciation of the significance of the experience when they start implementing what they are learning as part of their own research studies. It is recommended for such a research module, with a firm experiential learning component, to be integral to undergraduate medical programs. This is expected to improve the future physicians’ research competences, and in turn add value in terms of quality of care and patient outcomes.
Citation: Otaki F, AlHashmi D, Khamis AH, Azar AJ (2023) Investigating the evolution of undergraduate medical students’ perception and performance in relation to an innovative curriculum-based research module: A convergent mixed methods study launching the 8A-Model. PLoS ONE 18(1): e0280310. https://doi.org/10.1371/journal.pone.0280310
Editor: Anand Nayyar, Dai Hoc Duy Tan, VIET NAM
Received: March 11, 2021; Accepted: December 13, 2022; Published: January 13, 2023
Copyright: © 2023 Otaki et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: 8A-Model, The framework generated by this study to suggest that students go through four phases of an iterative process: Attend-Acquire, Accumulate- Assimilate, Apply-Appreciate, and Articulate-Affect, with an evident turning point around “Apply”; CoM, College of Medicine; CH, Credit Hours; GPA, Grade Point Average; IRB, Institutional Review Board; ANOVA, Analysis of Variance; MBBS, Bachelor of Medicine, Bachelor of Surgery, or in Latin: Medicinae Baccalaureus, Baccalaureus Chirurgiae; MBRU, Mohammed Bin Rashid University of Medicine and Health Sciences; SD, Standard Deviation; SPSS, Statistical Package for the Social Sciences; UAE, United Arab Emirates
Introduction
Early research training, particularly as part of undergraduate medical education, has been proven to significantly affect physicians’ career development [1–5] and professional identity [6–8]. There are several medical schools in the Middle East and North Africa region (MENA) that offer Bachelor of Medicine, Bachelor of Surgery (MBBS) programs. In those countries, MBBS is considered an equivalent to the Doctor of Medicine (MD). Admission into the MD in North America is usually restricted to those who already hold a university degree (e.g., a Bachelor of Medical Sciences). As for the MBBS program, the students join it right after completing their secondary education, and hence, it is considered an undergraduate program. Irrespective of the structure of the medical program, physicians are more likely to perform research as one of their primary professional activities if they are exposed to research experiences early-on in their educational trajectory [9–12]. For example, a study that investigated the impact of the Vanderbilt University School of Medicine Medical Scholars Program, showed that such programs prepare students for careers in academic medicine, and influences their career choices at an early juncture in their training [13]. Such experiences are associated with improved academic performance, and increased research interest and productivity [2, 14, 15]. Another study conducted in the Faculty of Medicine in the American University of Beirut revealed that group discussions and team-based learning are both effective instructional strategies to teach critical appraisal to undergraduate medical students [12].
The provision of quality medical care and the development of strong research skills, among healthcare practitioners, are inextricably linked [7, 16]. It is believed that research makes medical students better future clinicians and that it constitutes the core of the practice of medicine [1, 8, 17]. Research empowers the students to practice evidence-based medicine, enabling them to generate the necessary knowledge to reinforce decisions during residency and in their future practice [8, 18]. In a study aimed at exploring the students’ perception of a unique student-driven Undergraduate Research Committee at Alfaisal University, Riyadh, Saudi Arabia, showed that such committees provide the future generation of physicians with diverse training opportunities to pursue research careers [3]. In another study aimed at investigating the requirement of conducting a research project in the University of Sydney Doctor of Medicine Program, research supervisors reported that the major barrier to successful research projects was the lack of protected time for research activities. The assessment schedule with compulsory progress milestones enabled project completion. It also enhanced scientific presentation and writing skills. Supervisors highlighted the need for additional support for their students in running statistics, scientific writing, and identifying funding opportunities [5].
Embedding research curricula into undergraduate medical programs is still not widely adopted [2, 15, 16, 19], although it is globally recognized to be an integral component of physician training [3, 20, 21]. This is especially true if the learning opportunity is experiential in nature [22–25], and based on a holistic theory of education, which emphasizes learning as participation in the social world [26]. To the best of the authors knowledge, there seems to be a gap in the literature regarding the trajectory through which undergraduate medical students go through to learn applied research concepts. Accordingly, this study aimed at investigating the journey of undergraduate medical students in relation to a compulsory curriculum-based research module, which has a prominent experiential learning component. The research questions of this study are as follows:
- How do the MBBS students perceive the research module courses at different stages in their learning trajectory?
- How is the performance of the MBBS students evolving as they progress in the research module, and how does this trend relate to that of the same students’ performance across all courses in the respective semesters?
- What meta-inferences can be derived from integrating the qualitative data analysis (i.e., perception) with that of the quantitative one (i.e., performance)?
Materials and methods
Context of the study
This study was conducted at the Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU) in Dubai, United Arab Emirates (UAE). The College of Medicine (CoM), at MBRU, offers an undergraduate Bachelor of Medicine and Bachelor of Surgery (MBBS) program [27].
This MBBS program consists of a six-year curriculum, composed of three phases, built on a competency-based learning model. The learning process is spiral, with integrated courses through-out the six-years, totaling 12 semesters (ie 2 semesters per academic year) [28]. The first academic year, which constitutes Phase 1 of the respective MBBS program, exposes the students to basic concepts of medicine. Phase 2 covers the second and third academic years of the program, where teaching is organized around body organ systems and is integrated with clinical medicine. The fourth through sixth academic years represent Phase 3, through which the students undergo their clinical rotations, and wrap-up the program requirements with an internship [29]. The CoM received its first MBBS batch of students in August 2016 (ie Class of 2022).
Description of the research module under investigation
The research module is an integral part of the MBBS program at MBRU and is compulsory for all enrolled undergraduate medical students [30]. Those students transition from secondary school directly into the program without academic induction. This module consists of a series of five interrelated courses in epidemiology, biostatistics, and research methodology delivered over the first five consecutive semesters of the MBBS program, Table 1. These 5-integrated courses are in complete alignment with the MBBS Program Learning Outcomes at MBRU. Where, upon completing those 5-integrated courses, the student would have developed the competences needed for the third Program Learning Outcome: “practicing evidence-based medicine, and engaging in scholarship and generation of new knowledge” [27].
Each course builds upon the knowledge and skills obtained in the preceding course. The mode of delivery is sequential that reinforces the acquired knowledge and skills, among the students, as they progress in the research module. Eight Credit Hours (CH) are awarded for completing this module: the two consecutive courses offered in semesters 1 and 2 are each awarded 1 CH. As for the courses offered consecutively in semesters 3, 4, and 5, each are awarded 2 CH, Table 1.
The first three courses, delivered in Semesters 1 through 3 (totaling 4 CH), provide the students with a comprehensive background and understanding of the sciences of epidemiology and biostatistics, and of the basic technicalities of research methodology. The last two courses, delivered in Semesters 4 and 5 (totaling 4 CH), are more practical in nature, and are designed to further reinforce the students’ understanding of the principles of research design and methodology, where they apply what they learn through undergoing an actual research study, Table 1. This component of the learning experience is based on Situated Learning Theory, which is one of the holistic theories of education and emphasizes participation in the social world [26].
Each student performs their own research project, with an assigned supervisor, either basic or clinical sciences faculty member (depending on the scope of the study). The students are given an extensive list of health-related research opportunities of which they are given the autonomy to choose from. These projects are biomedical (clinical or lab-based) or socio-behavioral in nature; the projects in the latter track are related to health systems or medical education. The supervisors’ main responsibility is to co-create with their assigned supervisees, while mentoring them in relation to the subject matter throughout the scientific research process. As such, the individual students go through a process of adaption as they progress in the learning experience, which takes place in the authentic context, among the respective community of practice; all of which depends on what the individual student’s research project is.
The course learning objectives guide the instructional learning strategy of each course. The first three courses are delivered using both lecture-based teaching and case-based learning. The summative assessment contributes to the Grade Point Average (GPA) which is out of 4.0 and constitutes a direct measure of the individual students’ performance. The attainment of the learning objectives of these courses is measured by a combination of summative and formative assessments: Assignments (formative), Open-book Assessment (summative), and End-of-term Examination (summative). The course instructors are responsible for grading all the assessments.
As for the last two courses, they are delivered using both case-based teaching and project-based learning. The attainment of the learning objectives in semester 4 is measured by a combination of summative and formative assessments: Research Ethics Module (formative), Student Research Project Form (formative), Research Proposal (summative), In-course Assessment (summative), Student Progress (summative), and Oral Presentation (summative). As for semester 5, students are assessed on the following parameters: Conference Poster Presentation (summative), Research Project Dissertation (summative), Student Progress (summative), and Digital Abstract (summative). It is worth noting that the same group of instructors (three faculty members) are involved in delivering the 5-integrated courses and in grading all entailed assessments (using pre-defined rubric). The assessment plan, question type, and grading method used are congruent across all cohorts. Accordingly, consistency of assessment and corresponding grading scheme is ensured.
Research design
A convergent mixed methods study design was adapted to develop a systemic understanding of the experience of the undergraduate medical students throughout the respective research module. For that matter, qualitative and quantitative datasets were concurrently collected and in turn systematically integrated. The triangulation of data, as such, enables investigating the same phenomenon from differing perspectives, which in turn raises the validity of the generated findings. It is worth highlighting that this paper is written in accordance with mixed methods article reporting standards [31–33].
The qualitative segment of this study aimed at exploring the development of the perception of students in relation to the module, and to its immediate output, and foreseen outcome and long-term impact. It relied on a phenomenological research methodology [34, 35] based on the deployment of the Braun and Clarke (2006) six-step model of conducting thematic analysis [36]. This framework, which is based on a constructivist epistemology, is commonly used in the realm of social and behavioral research [37, 38].
In parallel, the performance of the students as they progress through each course of the research module was quantitatively analyzed. This quantitative segment relied on a cross-sectional time-series design [39], using the GPA of the students of all three classes (2022, 2023, and 2024) on an aggregate level. The performance of the students in each of the courses of the research module was measured. The generated trend across those courses was compared to that of the students’ performance of all courses (of the MBBS program) in the respective semesters.
Ethical approval
Ethical approval for the study was granted by the MBRU, Institutional Review Board (Reference # MBRU-IRB-2020-015). Prior to the commencement of the study, written informed consent was obtained from all participants.
Participants
The study was conducted in Fall 2019–2020 and involved three cohorts of MBBS students: Class of 2022 who had completed the entire research module, all 5 integrated courses (8 CH), Class of 2023 who had completed the first four courses (6 CH) out of 5 courses, and Class of 2024 who had completed the first two courses (2 CH) out of 5 courses, Table 2. Out of the respective MBBS students, 77% were female and 33% were UAE nationals. They were of 26 nationalities.
Data collection
Qualitative data.
The qualitative segment of this study involved exploring the evolution of the perception of the students regarding their experiences as part of the research module from joining the MBBS program. For that matter, three focus group sessions were conducted. Since the ideal size of a focus group is between five to eight participants [40, 41], seven randomly selected students, from each of the three classes (2022, 2023, and 2024), were invited to the focus group sessions (one corresponding to each class). The randomization was done using the ‘Select Cases’ function of SPSS statistical package (Windows version 25.0, IBM Corp, NY, USA).
A focus group protocol was tailor-made to guide the data collection initiative across all three sessions. Forty-five minutes were allocated for each group session, which was divided into four segments. The first 15 minutes inquired about the knowledge, skills, and competencies acquired through the module (including but not limited to: basic concepts and research process). The following 10 minutes were focused on the research learning experience, across the module, and the output, outcome, and impact of the module. The next 10 minutes included discussions around the effectiveness of the module, the students’ level of appreciation, and the value of research. The last 10 minutes constituted the wrap-up where the strengths and opportunities for improvement of the module were pinpointed. These focus group sessions were facilitated by a researcher experienced in designing and undergoing qualitative studies (FO).
The data collection tool, which was designed especially for this study, underwent two validation phases. Firstly, five faculty members at CoM were contacted for the content validity. Secondly, the questions of the tool were discussed with 10 randomly selected students to assess the readability and comprehensibility of the questions, and the sequence by which the questions were presented (ie face validity).
Students were informed that participation in the focus group sessions was completely voluntary and were given the option to withdraw from participation any time before or during the focus group session. In addition, the students were requested to provide written consent as a prerequisite to participation. Participating students were assured regarding the anonymity and confidentiality of the data generated from the respective sessions.
Each participant was given a unique identification number (ie participants were numbered 1 through 15). These unique identification numbers were complimented with ‘1’ for Class of 2024, ‘2’ for Class of 2023, and ‘3’ for Class of 2022 (ie participants 1 through 6 were followed by ‘1’, 7 through 11 by ‘2’, and 12 through 15 by ‘3’).
Quantitative data.
Assessment data, including but not limited to the GPA, is routinely gathered by the Student Services and Registration department at MBRU (SSR). Data on the students’ performance (GPA), per semester, across all courses offered in the respective semester, and in relation to each of the sequential courses of the research module were requested by the researchers from the SSR. The obtained GPA values corresponded to Semesters 1 through 5 (Class of 2022), Semesters 1 through 3 (Class of 2023), and Semester 1 (Class of 2024). This depended on where the respective students were in their learning trajectory. All the data related to the performance of the students was deidentified by a member of the SSR department prior handing it over to the research team.
Data analysis
Qualitative data.
The qualitative data was thematically analyzed by three researchers (AJA, DA, & AH), following the six steps of the abovementioned framework [36]. The data collected from each of the three cohorts was handled separately; the researchers were not informed which cohort each of the datasets corresponded to. The researchers started with familiarizing themselves with the data; patterns across the datasets were systematically identified and reflected upon. The second step included segmenting the data into meaningful statements and generating initial codes. NVivo software version 12 plus (QSR International Pty Ltd, Vic, Australia) was used to expedite the coding of the raw data; this facilitated the categorization of the relevant text fragments. The transcripts were examined, line-by-line, while coding the text fragments that relate to the research questions until no new information was observed in the data, and hence data saturation was attained. The third step included searching for themes. This led to the generation of a set of themes that refer to differing stages of the students’ learning journey. The themes were then reviewed as the fourth step, where they underwent several rounds of reflections.
The three researchers, as part of the fifth step, agreed on the optimal way to sequence those themes based on their collective understanding of the encapsulating context, delivered curriculum, and receiving students. The researchers factored into the analysis their interpretation of differing knowledge transfer, exchange, and valorization theories (eg Kirkpatrick model, Learning-Transfer Evaluation, system thinking, knowledge management, and processual analyses) [42–46]. They also coded the themes. As such, the conceptual framework of the study (with codes for all categories and themes) was developed.
All this paved the way for the last step of the adapted framework which constituted the basis of reporting upon the results and was done in alignment with recently published recommendations for reporting qualitative research [31–33].
With the intention of conducting a respondent validation [47, 48], the informant feedback was obtained through a discussion conducted after the completion of data analysis with the 4 participants of Class of 2022. To validate the themes and their sequence, the informants need to have completed the entire research module to be able to view the framework from a systemic/ macro perspective. Therefore, among the total 15 students who participated in the focus group sessions, only 4 students of the Class of 2022 were engaged in this validation step. In the respective meeting, the participants were shown the generated conceptual framework. After show-casing the conceptual framework and explaining it verbally to them, the students were given the space to reflect upon the extent of resonance between their perception of the research learning trajectory (that they have been through) and the conceptual framework. All students agreed with all the identified codes and how they relate to one another.
Quantitative data.
The extracted quantitative data was analyzed using the SPSS statistical package (Windows version 25.0, IBM Corp, NY, USA). Continuous data was described by measures of tendency and dispersion. Analysis of variance was used to compare the mean GPA in the different semesters. A p-value of less than 0.05 was considered statistically significant [49].
Mixed methods integration.
The findings from both types of analyses (qualitative and quantitative) were mapped onto each other and carefully reflected upon. This mixed methods integration took the form of an iterative process, namely: the joint display analysis [50]. This ultimately led to meta inferences [51]. The researchers explored how the output of the analyses relate to one another to synthesize a meaningful narrative.
Results
Qualitative
Out of twenty-one randomly selected students who were invited to the sessions, 15 (71%) agreed to participate: 4 (Class of 2022–1 female and 3 males), 5 (Class of 2023–5 females and 0 male), and 6 (Class of 2024–3 females and 3 males). The generated data included reflections of the research learning trajectory, from the perception of the students. As illustrated in the conceptual framework generated as part of this study (Fig 1), the thematic analysis depicts four sequential steps, namely: 1- Attend-Acquire, 2- Accumulate-Assimilate, 3- Apply-Appreciate, and 4-Articulate-Affect, that the students go through as part of the research module to effectively integrate the scientific research method, Table 3.
The qualitative component of this mixed methods research study generated this framework, depicting the different phases that students go through as part of the curriculum-based research module (represented as the Grey arrow). Each phase, representing a theme of the inductive analysis, appeared to be characterized by an interplay of two verbs. Hence, the code mindfully ascribed to each theme brings together two verbs, each starting with the letter ‘A’. The increasing size of circles represent the students’ accumulation of expertise (ie integrated knowledge and skills), and the color development (ie from light to dark) represents the gradual evolution that the students go through, over time.
Attend-Acquire.
The first theme encapsulates the text fragments that refer to the attendance of the students, which highlight the students’ enrolment, their in-class experience and contributions, and their interactions with the instructors. Some students reflect upon how they actively participate in their learning experiences as part of this course:
1.1. “…before we start the lecture, I go over the learning objectives, and as the instructor goes through the presentation explaining its content, I would be ticking off the learning objectives to confirm that we covered them all…”
The content of the first two courses is described as novel, and in some cases, confusing, to those who just started their journey. The interplay between theory and practice when it comes to the subject matter is apparent to the students from the beginning of the journey. Yet, the value of this duality, especially in relation to the biostatistics component, is not visible to them at the beginning. It took them time to attain this realization:
6.1. “…while the instructors are delivering the biostatistics content, we continuously shift between the theory and the practice using SPSS. It is difficult for us to see the link. We do not see how the theory translates into practice. We are told there are links, but we do not see them. This confuses us…”
7.2. “…we learned mostly about research. We started off the module learning about epidemiology…In parallel, they were teaching us biostatistics. We did not know back then the relevance and importance of biostatistics. Its true importance became apparent to us in Semester 4…”This theme also includes text fragments that relate to the knowledge and skills that the students acquire from their experience. The fact that there is an assortment of topics covered in the courses becomes clear to the student at this stage:
2.1. “…we learned about the different types of studies. For example: experimental and non-experimental studies. We will be using this knowledge to design our own research studies at a later stage…for biostatistics, it is all based on formulas. We need to practice it step-by-step to be able to understand and implement it in our research studies…”
Some students seem to notice the interconnectedness of the entailed topics. This holistic perspective might have been acquired (or at least exercised and in turn developed) through the public health component of the module:
5.1. “…we learned about public health…it is the art and science revolving around the health of the population…it shows you how it is all interlinked…the health and wellbeing of one patient is related to the status of the community that s/he belongs to…”
Besides the technical competencies acquired, the students highlight soft skills that they practice through their engagement in the course and with its content:
9.2. “…I am now better at writing professional emails, because there are so many emails that I had to send to my supervisors; there were a lot of correspondences, back and forth, between us…”
Accumulate-Assimilate.
The second theme is related to how the knowledge and skills are accumulating, and in turn shaping the students’ attitudes and habits (eg critical appraisal and retrieving evidence). It was clear from the transcripts that the learners have previous knowledge and experience. This existent base appears to constitute an increasing resource for learning, which takes the form of mental models. They also seem to deploy analogical reasoning in learning and practice. At this stage in the learning curve, the students start realizing the importance of the disciplines of biostatistics, epidemiology, and research, and of how they are interlinked. This appreciation seems to evolve over time. An affinity, of some sort, between the students and research publications starts growing at this stage:
3.1. “…this course enabled us to understand articles. We critically appraise articles. We gained the habit of screening data, information, and knowledge. We evaluate the quality of the evidence prior taking it into account…”
7.2. “…we now know how to properly read a peer-reviewed article. We do not just read the findings from the perception of the investigators. We investigate the numbers, the calculations, and the statistical techniques. We check the reliability of the study and the validity of the results. We critically-appraise its core, along with going through and reflecting upon the findings of the authors…”
8.2. “… in the beginning, we were given epidemiology and biostatistics. Back then, I did not understand neither their importance, nor their relevance to each other…I started seeing the link between them in the fourth semester…”
Moreover, the students mention that they start using what they are acquiring from the module across other courses within the MBBS program. In a way, the students were starting to realize that they were grasping tools that are deployable to reinforce learning within the module and beyond:
14.3. “… it is important to know how to effectively interpret data if one wants to implement evidence-based medicine…”
15.3. “…I started to use what I acquired from this module in other courses…the five courses turned out to be interlinked. …”
Then comes the stage, where the students seem to play an active role in assimilating, and in turn integrating the acquired skills and knowledge. They start building expertise and resilience:
1.1. “…knowledge and skills of research are very important, and go hand-in-hand with practicing medicine…as clinicians, we need to continue on reading articles, keeping an eye on new studies so we can stay up-to-date…”
The students appear self-directed and intrinsically motivated, and their learning experience is facilitated by skilled mentors. The learning appears to be occurring through social interactions:
11.2. “…my supervisor effectively mentored me to integrate all that I had been acquiring to ask the right questions. It was not an easy journey for me, but I made it through. It has been so enriching…”
Apply-Appreciate.
Following that, the students start applying what they have been acquiring, accumulating, and assembling. The learners appear to be continuously reflecting upon these concrete, hands-on experiences, which are taking place in safe environments. They are accordingly adapting their mental models and are engaging in active experimentation:
7.2. “…for you to know something very well, you would have to do it…”
The students feel empowered and a sense of ownership while undergoing their own research study. As such the experiential learning component of this study is providing an opportunity for the learners to try-out new ideas, and in turn cement this knowledge and long-term changes in practice:
9.2. “…I used to always hear about research and physicians doing research, and that would get me really excited, but I did not know what research is all about…Now, I am informed, and feel empowered to attain my aspirations…”
After the students start putting into practice what they have been learning, they start appreciating the journey and all that it entails. The students express gratitude, and in some cases- excitement:
9.2. “…we are grateful that we are given this opportunity at an early stage in our educational journey…”
14.3. “…this module made me appreciate research and how research is done. I really enjoy the scientific research process. I felt a strong sense of accomplishment upon completing and submitting my research study…I am now involving myself in other research opportunities…”
Some students go all the way to appraising their proficiency in the acquired skills:
12.3. “…my research topic was a literature review. I am now proficient in running systematic literature reviews. This is helping me during my internships, and as I am studying for other courses…”
Articulate-Affect.
The last theme includes reflections regarding how the students are articulating their findings and contributing to the theory (and practice) of the subject matter (be it through presenting and/ or publishing), which they are convinced would enrich their professional profiles.
9.2. “…we get a lot of opportunities to share, present, and collectively reflect upon our work…it is good to indicate this experience on our professional profiles…”
13.3. “…I learned what it takes to publish an article, and that, as they say: ‘the whole is more than the sum of its parts’…”
Their experiences, as part of this module, appear to have raised their self-awareness and -actualization.
13.3. “…this module enabled me to understand the importance of research and practicing evidence-based medicine…”
14.3. “…I learned how to design…I designed a research study and wrapped-up the experience with designing a poster presentation and a conference digital abstract video. You need to learn how to ask the right questions, and to be equipped with the knowledge and skills to answer them effectively…”
This theme also relates to how the students perceive themselves to be affecting and altering the field by practicing evidence-based medicine, improving performance (clinical or otherwise), and developing communities. They also refer to how they are leveraging the expertise that they acquired in this module to coach others:
7.2. “…this is particularly relevant to our region. By conducting research and generating knowledge, we will be contributing to the development of our countries…”
14.3. “…I learned a lot from the entire research experience…I am using what I gained from these courses in my internships…”
Quantitative
Sample description.
The total number of students in all 3 classes (2022, 2023, and 2024), whose GPA values were factored into the analyses, was 158. Yet, the number of students per semester differed as this was dependent, by design of the study, on where the respective students were in their learning trajectory: Class of 2022 (54 students) who had completed the entire research module; Class of 2023 (38 students) who completed the first 4 Semesters; and Class of 2024 (66 students) who completed the first two semesters. In other words, 158 students completed Semesters 1 and 2, 92 completed Semesters 3 and 4, and 54 completed Semester 5.
Research module.
The quantitative analysis resulted in two distinct clusters of mean GPA values across the 5-integrated courses’ module given in semesters 1 through 5, as shown in Fig 2. The first cluster was for courses delivered in semesters 1, 2, and 3 where the mean GPA values were found to be homogenous (p>0.05). The second cluster, which was also established to be homogeneous (p>0.05), was for the mean GPA values of students across the courses offered in semesters 4 and 5. Moreover, the mean GPA values for Semesters 1, 2, and 3, as compared to Semester 4 and 5, was significantly lower in the former cluster (p<0.01). The same trend, with a similar significance, was observed when the data of Semester 1 to 5 of Class of 2022 only was analyzed. It became evident that two significantly different clusters were observed cluster 1 (Semesters 1, 2, and 3) and cluster 2 (Semesters 4 and 5) (P<0.01). Accordingly, there appeared to be an evident turning point in the performance of the students, within the research module, in between clusters 1 and 2 (i.e., in between semesters 3 and 4).
Blue lines- There are two distinct clusters of mean GPA values across the 5-integrated course modules: first cluster was for courses of semesters 1, 2, and 3, and the second cluster was for courses of semesters 4 and 5. The 95% CI for Semester 1, 2, and 3 overlap indicating no statistical difference between the three research courses given in the first 3 semesters. Similarly, the 95% CI for the research courses given in Semester 4 and 5 overlap indicating no statistical difference (p>0.05). However, post-hoc ANOVA analysis revealed two distinct clusters of mean GPA, the first cluster (Semesters 1, 2, and 3) and the second cluster (Semesters 4 and 5) (p<0.01).
All program courses.
As evident in Fig 2, the mean GPA values of the students, across all courses offered as part of the MBBS program (including the courses of the research module), was heterogeneous. The results showed that the mean GPA in semester 1 was significantly different than that of semesters 2, 3, and 5 (p<0.01); semester 2 was significantly different than that of semesters 1 and 5 (p<0.01); semester 3 was significantly different than that of semesters 1 and 4 (p<0.01); semester 4 was significantly different than that of semesters 1 and 2 (p<0.01); and semester 5 was significantly different than that of semester 3 (p<0.01). Indicating the absence of any clustering. Accordingly, there appeared to no obvious trends or patterns in the performance of the students across all program courses (while they are partaking in the research module).
Mixed methods integration
Merging the output of the thematic analysis into that of the quantitative analysis unearthed a holistic perspective of the situation, illustrated in the study’s joint display (Fig 3). The convergence of findings enabled the development of a thorough understanding of how the students’ perception and performance evolve as they progress through the module. On its own, the narrative analysis showed the 4 steps that the students go through as part of the journey, with particular emphasis on the value of the step where the students start putting into practice what they have been learning. As for the quantitative analysis (on its own), it revealed that the students’ performance can be observed as two distinct clusters. These findings confirmed the results of the qualitative segment of this study, illustrating that the turning point in the learning trajectory of the students is at the “Apply” step of the introduced conceptual framework.
Mapping the study’s conceptual framework onto the student’s performance revealed a holistic perspective of the situation. The convergence of findings, as such, confirmed the existence of a turning point (highlighted with a Yellow dotted line). The students started expressing appreciation of the value of the learning experience upon putting into practice the knowledge and skills that they were acquiring, accumulating, and assimilating. The color transformation [ie from a primary color (ie Blue) to a secondary color (ie Green: Blue + Yellow)] further emphasizes this turning point, and how the increased appreciation happened concurrently with their increased engagement and enhanced performance.
Discussion
This study introduced the evidence-driven 8A-Model. This framework, in alignment with the Kirkpatrick’s four level model [52, 53] and other theories around knowledge acquisition [54–56], suggests that the undergraduate medical students enrolled in an integrated research module go through specific milestones in order to effectively integrate the scientific research method. These steps start with students attending the courses, followed by acquiring and in turn accumulating the knowledge and skills. Next the students assimilate what they are acquiring, which in turn, enables them to apply what they have been learning all along. This empowers the students, solicits their appreciation, and increases their engagement (as reflected in their enhanced performance). It then encourages them to articulate their findings, which in turn affect the fields of medicine and/ or public health, instilling constructive change and improvements. It is worth highlighting that the male and female participants, in the current study, appeared to reflect similarly on their research learning trajectory. Moreover, the gender of the students appeared to be independent of their performance in the respective research module. Hence, in alignment with some previously conducted research on the subject matter [57, 58], we conclude that the gender of the participants in the current study did not seem to play a significant role in the process of integrating research knowledge, skills, and attitudes. This might be due to the firm standardization around the students’ admission process of all medical schools, in general, and MBRU, in specific, where the effectiveness of the learning and teaching are greatly dependent on having all the admitted students start their learning trajectory, on the same level in terms of academic performance and a selection of competences.
In parallel, the quantitative analysis revealed that the learning curve integral to the trajectory of the students, as part of the research module, can be observed as two segments. From the first through the third course, the three averages of performance of the students were similar. The averages of performance significantly increased in the fourth and fifth courses. Congruent with the results of the qualitative segment of this study, the turning point is at the “Apply” step of the introduced framework. As suggested, in previous studies, it is not enough to teach students about research [59]. Students need to be provided with the opportunities to conduct research, and to be counselled on the attitudes needed for them to thrive in research environments [59, 60]. This is especially true when, similar to the case of the research module under investigation in this study, the experiential learning component is based on the Situated Learning Theory [26, 61], and when the educator considers individuals, and their experiences and environments [62, 63]. As such, the experiential education was maximized through the individual students’ embeddedness in the authentic context, among a community-of-practice. Their active adaption is facilitated by experts in the subject matter. Accordingly, it is recommended for other undergraduate medical programs to adapt similar research modules, with a concrete experiential learning component. Learners in such programs need to be enabled to deploy analogical reasoning throughout their experiences, and to continuously reflect upon their experiences while developing their existent mental models [63].
Along these lines, this study showed that the students start to truly appreciate the value of the learning experience upon using the knowledge and skills that they are acquiring, accumulating, and assimilating. This appreciation is reflected in their increased engagement and enhanced performance. It is worth noting that this observation is quite different from the suggestions of other traditional knowledge transfer and integration models (eg Kirkpatrick model and the Learning-Transfer Evaluation) which highlight appreciation (or the lack of it) as an instant result of participating in any one learning experience [43, 53]. This discrepancy may be associated with the fact that the module under investigation is integral to an MBBS program (as opposed to an MD program), where the students start their research learning trajectory right upon completing their secondary education. When it comes to effectively integrating the scientific research method, there is a prominent learning curve with a gradual evolution, and the content acquired is usually novel. So, it takes time to properly digest the acquired content, and start putting it into practice. All this is expected to ultimately offer value to patients, while lowering costs of health care. The offered value, in this case, will be driven by healthcare providers experience and expertise [64, 65].
Moreover, the results show that the performance of the students throughout the research module is more consistent, with less fluctuations, than their performance across all courses in the first five semesters of the MBBS program. This could be indicative of the level of integration within each course of the research module, and across all five courses of the respective module. It is worth highlighting that although the 8A-Model is diagrammatically depicted as a linear process. It is more likely to be an iterative one given the required spiral mode of delivery which includes purposeful repetition of specific content to ensure effective integration within each course and across all five courses.
The convergent mixed methods study design enabled the development of thorough insights into this innovative research module, and its application as part of an undergraduate medical program. Yet, the generalizability of the results is limited to contexts that are like MBRU. Hence, it is worthwhile for future studies to investigate the application of such modules, and the validity and reliability of the generated 8A-Model across multiple medical programs. Moreover, by virtue of the selected study design no causality can be established. It would be worthwhile for future studies to investigate the same variables (ie perception and performance) longitudinally (preferably through an experimental design) to better understand how the respective variables relate to one another. It will also be worth conducting a follow-up longitudinal study to investigate the long-term outcome and impact of such a curriculum-based intervention. Dependent variables such as: research interest and productivity, likelihood of pursual of research-based careers, and engagement in evidence-based medicine can be investigated. In terms of creating value-based health care, the influence of such an intervention on the quality of care and patient outcomes can be examined.
Conclusion
The evidence-driven 8A-Model, generated by this study, highlights that students’ understanding of the true value of research seems to increase as they progress in the module. They begin expressing appreciation of the significance of the experience when they start implementing what they are learning (acquiring, accumulating, and assimilating) as part of their own research studies. It is recommended for such a research module, with a firm experiential learning component, to be integral to undergraduate medical programs. This is expected to improve the research behaviors, skills, and attitudes of the students on the short run. In terms of long-term impact of such an intervention, it is believed to add value in terms of quality of care and patient outcomes.
Acknowledgments
The corresponding author would like to extend gratitude to Dr. Tom Loney for his valuable contribution to delivering components of the research module under investigation as part of this study.
References
- 1. Langhammer CG, Garg K, Neubauer JA, Rosenthal S, Kinzy TG. Medical student research exposure via a series of modular research programs. J Investig Med. 2009;57(1):11–7. pmid:19092679
- 2. Amgad M, Man Kin Tsui M, Liptrott SJ, Shash E. Medical Student Research: An Integrated Mixed-Methods Systematic Review and Meta-Analysis. PLoS One. 2015;10(6):e0127470. pmid:26086391
- 3. Alamodi AA, Abu-Zaid A, Anwer LA, Khan TA, Shareef MA, Shamia AA, et al. Undergraduate research: an innovative student-centered committee from the Kingdom of Saudi Arabia. Med Teach. 2014;36 Suppl 1:S36–42. pmid:24617783
- 4. Peacock J. G. WLL, Drozdowicz L. B., Martin B. A., Suresh R., Denzer B. J., Wentworth A. B., et al. Curricular Flexibility in the Pre-Clinical Years Promotes Medical Student Scholarship. Medical Science Educator volume. 2013;23:92–8.
- 5. Hart J, Hakim J, Kaur R, Jeremy R, Coorey G, Kalman E, et al. Research supervisors’ views of barriers and enablers for research projects undertaken by medical students; a mixed methods evaluation of a post-graduate medical degree research project program. BMC Med Educ. 2022;22(1):370. pmid:35562832
- 6. Rosenberg L. Physician-scientists—endangered and essential. Science. 1999;283(5400):331–2. pmid:9925491
- 7. Weston WW. Do we pay enough attention to science in medical education? Can Med Educ J. 2018;9(3):e109–e14. pmid:30140355
- 8. Detsky ME, Detsky AS. Encouraging medical students to do research and write papers. CMAJ. 2007;176(12):1719–21. pmid:17548386
- 9. Hunskaar S, Breivik J, Siebke M, Tommeras K, Figenschau K, Hansen JB. Evaluation of the medical student research programme in Norwegian medical schools. A survey of students and supervisors. BMC Med Educ. 2009;9:43. pmid:19602226
- 10. Toledo P, McLean S, Duce L, Wong CA, Schubert A, Ward DS. Evaluation of the Foundation for Anesthesia Education and Research Medical Student Anesthesia Research Fellowship Program Participants’ Scholarly Activity and Career Choices. Anesthesiology. 2016;124(5):1168–73. pmid:26928347
- 11. Peacock JG, Grande JP. A flexible, preclinical, medical school curriculum increases student academic productivity and the desire to conduct future research. Biochem Mol Biol Educ. 2015;43(5):384–90. pmid:26148309
- 12. Daou D, Chakhtoura M, El-Yazbi A, Mukherji D, Sbaity E, Refaat MM, et al. Teaching critical appraisal to large classes of undergraduate medical students using team-based learning versus group discussions: a randomized controlled trial. BMC Med Educ. 2022;22(1):77.
- 13. Brown AM, Chipps TM, Gebretsadik T, Ware LB, Islam JY, Finck LR, et al. Training the next generation of physician researchers—Vanderbilt Medical Scholars Program. BMC Med Educ. 2018;18(1):5. pmid:29301521
- 14. Brown AM, Morrow JD, Limbird LE, Byrne DW, Gabbe SG, Balser JR, et al. Centralized oversight of physician-scientist faculty development at Vanderbilt: early outcomes. Acad Med. 2008;83(10):969–75. pmid:18820531
- 15. Samarth S. SA, Neema A., Satyam S., Paridhi D. Inculcating research methodology related skills and aptitude amongst medical undergraduates- An interventional study. Journal of Family Medicine and Primary Care. 2022;11:3648–53.
- 16. Muhandiramge J, Vu T, Wallace MJ, Segelov E. The experiences, attitudes and understanding of research amongst medical students at an Australian medical school. BMC Med Educ. 2021;21(1):267. pmid:33971858
- 17. Ommering BWC, van Blankenstein FM, van Diepen M, Dekker FW. Academic Success Experiences: Promoting Research Motivation andSelf-Efficacy Beliefs among Medical Students. Teach Learn Med. 2021;33(4):423–33. pmid:33632042
- 18. Romero-Robles MA, Soriano-Moreno DR, Garcia-Gutierrez FM, Condori-Meza IB, Sing-Sanchez CC, Bulnes Alvarez SP, et al. Self-perceived competencies on evidence-based medicine in medical students and physicians registered in a virtual course: a cross-sectional study. Med Educ Online. 2022;27(1):2010298. pmid:34919030
- 19. Assar A, Matar SG, Hasabo EA, Elsayed SM, Zaazouee MS, Hamdallah A, et al. Knowledge, attitudes, practices and perceived barriers towards research in undergraduate medical students of six Arab countries. BMC Med Educ. 2022;22(1):44. pmid:35042492
- 20. Murdoch-Eaton D, Drewery S, Elton S, Emmerson C, Marshall M, Smith JA, et al. What do medical students understand by research and research skills? Identifying research opportunities within undergraduate projects. Med Teach. 2010;32(3):e152–60. pmid:20218832
- 21. Kupfer DJ, Hyman SE, Schatzberg AF, Pincus HA, Reynolds CF 3rd,. Recruiting and retaining future generations of physician scientists in mental health. Arch Gen Psychiatry. 2002;59(7):657–60. pmid:12090819
- 22. Hill B. Research into experiential learning in nurse education. Br J Nurs. 2017;26(16):932–8. pmid:28880621
- 23. Fredericks L, Koch PA, Liu AA, Galitzdorfer L, Costa A, Utter J. Experiential Features of Culinary Nutrition Education That Drive Behavior Change: Frameworks for Research and Practice. Health Promot Pract. 2020;21(3):331–5. pmid:32011916
- 24. de Oliveira SN, do Prado ML, Kempfer SS, Martini JG, Caravaca-Morera JA, Bernardi MC. Experiential learning in nursing consultation education via clinical simulation with actors: action research. Nurse Educ Today. 2015;35(2):e50–4. pmid:25563657
- 25. Afshari M, Witek NP, Galifianakis NB. Education Research: An experiential outpatient teleneurology curriculum for residents. Neurology. 2019;93(4):170–5. pmid:31332085
- 26. Quay J. Experience and Participation: Relating Theories of Learning. The Journal of Experiential Education. 2003;26(2):105–16.
- 27.
CER CaERd. Discover your potential at Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU) Dubai, United Arab Emirates2016 [University Website]. Available from: https://www.mbru.ac.ae/.
- 28. Banerjee Y, Tuffnell C, Alkhadragy R. Mento’s change model in teaching competency-based medical education. BMC Med Educ. 2019;19(1):472. pmid:31882006
- 29. Otaki F, Zaher S, Du Plessis S, Lakhtakia R, Zary N, Inuwa IM. Introducing the 4Ps Model of Transitioning to Distance Learning: A convergent mixed methods study conducted during the COVID-19 pandemic. PLoS One. 2021;16(7):e0253662. pmid:34264968
- 30. Azar AJ KA, Naidoo N, Lindsbro M, Boukhaled JH, Gonuguntla S, Davis D, et al. Design, Implementation and Evaluation of a Distance Learning Framework to Expedite Medical Education during COVID-19 pandemic: A Proof-of-Concept Study. Journal of Medical Education and Curricular Development. 2021;Volume 8:1–16. pmid:35392266
- 31. Levitt HM, Bamberg M, Creswell JW, Frost DM, Josselson R, Suarez-Orozco C. Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report. Am Psychol. 2018;73(1):26–46. pmid:29345485
- 32. O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–51. pmid:24979285
- 33. Seaman JD U.; Humberstone B.; Martin B.; Prince H.; Quay J. Joint Recommendations on Reporting Empirical Research in Outdoor, Experiential, Environmental, and Adventure Education Journals. Journal of Experiential Education. 2020.
- 34. Wong LP. Focus group discussion: a tool for health and medical research. Singapore Med J. 2008;49(3):256–60; quiz 61. pmid:18363011
- 35. Varkey P, Peloquin J, Reed D, Lindor K, Harris I. Leadership curriculum in undergraduate medical education: a study of student and faculty perspectives. Med Teach. 2009;31(3):244–50. pmid:18825566
- 36. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006;3(2):77–101.
- 37. Amir Rad FA, Otaki F, AlGurg R, Khan E, Davis D. A qualitative study of trainer and trainee perceptions and experiences of clinical assessment in post-graduate dental training. Eur J Dent Educ. 2020. pmid:32813939
- 38. Kiger ME, Varpio L. Thematic analysis of qualitative data: AMEE Guide No. 131. Med Teach. 2020;42(8):846–54. pmid:32356468
- 39. Velicer WF, McDonald RP. Cross-Sectional Time Series Designs: A General Transformation Approach. Multivariate Behav Res. 1991;26(2):247–54. pmid:26828253
- 40. Tavakol M, Sandars J. Quantitative and qualitative methods in medical education research: AMEE Guide No 90: Part I. Med Teach. 2014;36(9):746–56. pmid:24846122
- 41. Tavakol M, Sandars J. Quantitative and qualitative methods in medical education research: AMEE Guide No 90: Part II. Med Teach. 2014;36(10):838–48. pmid:24845954
- 42. Frye AW, Hemmer PA. Program evaluation models and related theories: AMEE guide no. 67. Med Teach. 2012;34(5):e288–99. pmid:22515309
- 43. Moreau KA. Has the new Kirkpatrick generation built a better hammer for our evaluation toolbox? Med Teach. 2017;39(9):999–1001. pmid:28649887
- 44. Webb AM, Tsipis NE, McClellan TR, McNeil MJ, Xu M, Doty JP, et al. A first step toward understanding best practices in leadership training in undergraduate medical education: a systematic review. Acad Med. 2014;89(11):1563–70. pmid:25250751
- 45. Ali K, Cockerill J, Bennett JH, Belfield L, Tredwin C. Transfer of basic science knowledge in a problem-based learning curriculum. Eur J Dent Educ. 2020;24(3):542–7. pmid:32343029
- 46. Brousselle A, Contandriopoulos D, Lemire M. Using Logic Analysis to Evaluate Knowledge Transfer Initiatives: The Case of the Research Collective on the Organization of Primary Care Services. Evaluation (Lond). 2009;15(2):165–83. pmid:23543910
- 47. Mascarenhas S. A-HM, Otaki F., Nasaif M., Davis D. Simulation-based education for selected communication skills: exploring the perception of post-graduate dental students. Korean journal of medical education. 2021;33(1). pmid:33735553
- 48. Triangulation H. T., respondent validation, and democratic participation in mixed methods research. Journal of mixed methods research. 2012;6(2):111–23.
- 49.
Bernard R. Fundamentals of Biostatistics. 8th ed. ed. Brooks/Cole CL, editor2016.
- 50.
Fetters MD. The Mixed Methods Research Workbook: Activities for Designing, Implementing, and Publishing Projects (Mixed Methods Research Series) 1st Edition. 2020.
- 51. Guetterman TF M.D.; Creswell J.W. Integrating Quantitative and Qualitative Results in Health Science Mixed Methods Research Through Joint Displays. The Annals of Family Medicine. 2015;13(6):554–61. pmid:26553895
- 52.
Kirkpatrick D KJ. Evaluating training programs: The four levels.: Berrett-Koehler Publishers; 2006.
- 53. Thalheimer W. The Learning-Transfer Evaluation Model (LTEM) 2018 [Available from: https://www.worklearning.com/2018/02/14/the-learning-transfer-evaluation-model-ltem/.
- 54. Rukavina I, Daneman M. Integration and its effect on acquiring knowledge about competing scientific theories for text. Journal of Educational Psychology. 1996;88(2):272–87.
- 55. Schmidt HG, van der Arend A, Moust JH, Kokx I, Boon L. Influence of tutors’ subject-matter expertise on student effort and achievement in problem-based learning. Acad Med. 1993;68(10):784–91. pmid:8397613
- 56. Schmidt HG, Boshuizen H.P.A. On Acquiring Expertise in Medicine. Educational Psychology Review. 1993;5(3):205–21.
- 57. Jenkins MR, Herrmann A, Tashjian A, Ramineni T, Ramakrishnan R, Raef D, et al. Sex and gender in medical education: a national student survey. Biol Sex Differ. 2016;7(Suppl 1):45. pmid:27785347
- 58. Meyer JH DT, Richardson JT. A gender comparison of contextualised study behaviour in higher education. Higher Education. 1994(27(4)):469–85.
- 59. Mabvuure NT. Twelve tips for introducing students to research and publishing: a medical student’s perspective. Med Teach. 2012;34(9):705–9. pmid:22905656
- 60. Mabvuure NT, Rodrigues J, Cumberworth A, Mahmud M. Twelve tips for running successful junior doctor-led teaching programmes for medical students. Med Teach. 2013;35(8):628–32. pmid:23544916
- 61. Seaman JB M.; Quay J. The Evolution of Experiential Learning Theory: Tracing Lines of Research in the JEE. Journal of Experiential Education. 2017.
- 62. Norcini JJ, Banda SS. Increasing the quality and capacity of education: the challenge for the 21st century. Med Educ. 2011;45(1):81–6. pmid:21155871
- 63. Zigmont JJ, Kappus LJ, Sudikoff SN. Theoretical foundations of learning through simulation. Semin Perinatol. 2011;35(2):47–51. pmid:21440810
- 64. Putera I. Redefining Health: Implication for Value-Based Healthcare Reform. Cureus. 2017;9(3):e1067. pmid:28409068
- 65. Baumhauer JF, Bozic KJ. Value-based Healthcare: Patient-reported Outcomes in Clinical Decision Making. Clin Orthop Relat Res. 2016;474(6):1375–8. pmid:27052020