A Mixed Methods Evaluation of a Purposefully Designed Short, Intensive Workshop-Style Course for Teaching Mixed Methods Research in Taiwan
Hou S-I and Chou W-Y
Published on: 2025-08-11
Abstract
Introduction: Mixed methods research (MMR) is increasingly important in graduate education, yet teaching it effectively in short, intensive formats is challenging. This study evaluates a two-day, workshop-style MMR course designed to build foundational skills in a culturally and linguistically diverse context.
Purpose: To examine how course design supports student learning, identify effective instructional elements, and explore factors influencing engagement and perceived impact.
Methods: A qualitative-dominant mixed methods process evaluation was conducted across four cohorts (N = 65) at two large national universities in Taiwan (2022–2025). Baseline, mid-course, and end-course surveys collected Likert-scale ratings and rich qualitative reflections. Joint display analyses integrated quantitative and qualitative data to identify convergence and emergent themes.
Results: All core course components were rated highly at end-course (means 4.48–4.64/5). Qualitative themes highlighted the value of instructor guidance (33%), interactive learning (28%), and authentic application (26%) for deepening understanding. Student challenges included English comprehension (37%) and limited methodological background (27%), mitigated by instructional approach and instructor support, independent study, and peer learning. Post-course reflections showed reduced worry (38%), increased MMR confidence (36%), and intent to apply MMR in future research (34%). Positive feedback across cohorts emphasized the instructor’s role in bilingual delivery, active learning, and responsive teaching.
Discussion: Thoughtfully designed, multimodal MMR short courses can effectively build competence in compressed timeframes. Integrated, culturally responsive instruction and active learning strategies are critical to success. Mixed methods process evaluation provided actionable insights into learning mechanisms and modelled MMR as both course content and an evaluation framework.
Keywords
Mixed Methods Research; Course Design; Intensive Short Course; Active Learning; TaiwanIntroduction
Mixed Methods Research in Education
Mixed methods research (MMR) has gained prominence in educational inquiry by combining the measurable patterns of quantitative data with the rich, contextual insights of qualitative data [1,2]. Effective integration requires more than simply collecting both types of data; it demands methodological rigor and thoughtful design to yield meaningful, cohesive results [2]. Recent applications emphasize inclusive, evidence-based teaching practices that foster relational classroom climates and equitable student outcomes [3]. Reflecting this momentum, the prevalence of MMR studies in educational journals has risen from 0.64% in 2011 to nearly 3% in 2024 [4]. By enhancing validity through triangulation, MMR addresses complex educational phenomena that require both quantitative measurement and qualitative contextualization. As interest in MMR grows across disciplines, there is a pressing need for pedagogical strategies that develop both conceptual understanding and applied skills in this methodology [5].
Challenges in Teaching Mixed Methods Research
Teaching MMR is inherently challenging due to its complexity and the need to integrate two distinct methodological traditions-quantitative and qualitative-while navigating underlying philosophical differences [6]. Traditional curricula typically separate these strands, leaving students with fragmented knowledge and limited readiness for authentic mixed methods research [7]. Addressing this gap requires intentionally designed courses that emphasize integration, active learning, and practical application rather than passive content delivery [8,9].
Short, intensive course formats-such as two-day workshops-have gained popularity in graduate education as a time-efficient approach for students and professionals. These immersive formats allow focused, applied learning, making them particularly appealing for complex topics like MMR. However, despite their growing use, there is limited empirical evaluation of their effectiveness in developing methodological competencies, highlighting the need for further investigation into best practices and outcomes.
Process Evaluation as a Framework
Process evaluation offers a valuable lens for understanding not only what students learn but also how learning occurs, how the course is delivered, and which contextual factors influence outcomes [10-13]. This approach provides insights into the mechanisms that drive course effectiveness, helping educators identify what works, for whom, and under what conditions-particularly important when assessing innovative or non-traditional pedagogical models.
Teaching mixed methods research (MMR) requires more than content delivery; it involves aligning course objectives, instructional strategies, and assessments to help students bridge methodological paradigms, appreciate the value of integration, and develop confidence in applying MMR to real-world research [3,8,9,14]. Despite growing interest in MMR pedagogy, empirical evidence on effective teaching strategies-especially for short-course formats-remains limited [4]. Most existing studies focus on curriculum content rather than delivery or instructional design, leaving educators with little guidance for teaching complex methodologies within condensed timeframes.
MMR Momentum in Asia
Mixed methods research has been gaining momentum across Asia, with increasing collaboration among scholars from China, Korea, Japan, Taiwan, and Thailand. A milestone event was the inaugural Asia Open Forum (AOF) in 2019, hosted by the Japan Society for Mixed Methods Research (JSMMR), which brought together regional leaders and was hailed by Dr. John Creswell as a “historical moment” for advancing MMR in Asia [15]. Since then, Taiwan has seen substantial growth in MMR education and practice, supported by workshops and intensive courses at leading universities and driven by mentorship and encouragement from international MMR pioneers such as Dr. Michael Fetters [7,15]. These efforts have laid the groundwork for building sustainable MMR training pathways and fostering regional collaboration in methods education.
Study Purpose
This study conducted a mixed methods process evaluation of an intensive, two-day foundational mixed methods research (MMR) course to address gaps in evidence on effective short-course pedagogy. The evaluation examined how the course design supported student learning, identified the most effective instructional elements, and explored factors influencing engagement and comprehension. Using a mixed methods approach to evaluate an MMR course aligns the study’s methodology with its subject, providing practical insights and contributing to the growing literature on methods education [8,9].
The implications extend beyond a single course or institution. As global adoption of MMR grows, well-designed training is essential to enhance research quality, foster interdisciplinary collaboration, and advance evidence-based practice. Findings from this study offer actionable guidance for educators and program developers, helping prepare researchers for the complex, integrative work that mixed methods research entails.
Methods
Course Development and Implementation
The Mixed Methods Research (MMR) Short Course is a one-credit, two-day intensive course designed to introduce modern MMR methodology to interdisciplinary master’s and doctoral students in public health and the social sciences. The course originated in late 2021 during the first author’s visit to Taiwan, when faculty and student interest in MMR training coincided with the university’s English-Mediated Instruction (EMI) initiative. These discussions led to the development of a new elective course delivered in English.
Following administrative approval, the course was first offered virtually in fall 2022 during the COVID-19 pandemic at a large national university in northern Taiwan and was well received, prompting requests for future in-person sessions. In 2023, a second national university invited the same instructor to teach the course, later adopting it as an approved elective for its graduate public health program [15]. To date, the course has been delivered to four cohorts-one virtual (fall 2022) and three in-person (summer 2023, fall 2023, summer 2025)-across the two universities in Taiwan.
The course was adapted from an original semester-long, three-credit doctoral seminar taught in the United States into a one-credit format for a broader graduate audience [8,9]. The redesign retained core theoretical foundations and hands-on applications, structured for workshop-style delivery. Course content emphasized conceptual understanding, applied learning, and critical engagement with MMR, aligning with the increasing need to integrate quantitative and qualitative approaches in public health, education, and social science research.
Course Purpose
The purpose of the MMR short course was to provide graduate students with a foundational understanding of modern mixed methods research and its applications to complex research questions in public health and related fields. Students were expected to articulate the rationale for using MMR, identify core design types, apply key terminology in evaluating published studies, and critically examine empirical applications of MMR to social and behavioral science interventions.
The course objectives were aligned with MPH foundational competencies as outlined by the Council on Education for Public Health [16,17], including cross-cutting skills and Social and Behavioral Sciences (SBS) concentration competencies. Specifically, the course supported students in (1) describing theories, concepts, and models from social and behavioral sciences relevant to public health and (2) applying evidence-based approaches to the development and evaluation of SBS interventions.
Course Design Features
Given the short, intensive format, the course emphasized active learning and structured engagement. Students participated in five in-class activities: (1) identifying MMR designs, (2) locating and reviewing MMR articles, (3) discussing the purpose of mixing methods, (4) conducting a team-based case study analysis, and (5) guided reflections on learning. Two mini-assignments extended learning outside the classroom: a review of MMR core features and a team-based case study report.
Instructional design drew on evidence-based strategies developed in the original three-credit MMR course [8,9]. Key elements included interactive lectures with Q&A, in-class activities with feedback, mini assignments to reinforce content, case study analysis to illustrate design application, and article reviews to develop critical appraisal skills. The course used Designing and Conducting Mixed Methods Research [1] as the primary text.
Course activities incorporated real-world MMR examples, such as a sophisticate mixed methods cancer screening study among Chinese women [7,8,30,31], to illustrate exploratory sequential, concurrent, and embedded evaluation designs. Students were encouraged to discuss methodological alternatives and reflect on practical applications. As the only MMR course in Taiwan at the time, it was viewed as highly valuable for developing foundational competencies in an emerging research area [7-9,15,18]. Table 1 summarizes the MMR course topics and assignment activities, along with their alignment to the MMR Competency Scale (MMRCS_8) performance measures, using a condensed version adapted from the original research-validated tool [9].
Table 1: MMR Course Topics, Assignment Activities, and MMRCS_8 Performance Competency Measurement.
|
Session |
Topic / Activity |
MMRCS_8 Measures* |
|
1-2 |
Course overview & welcome Introduction to mixed methods research |
MMRCS-1 (Define what is mixed methods research (MMR) and its core characteristics) |
|
3-4 |
Activity #1 – Identify MMR designs Activity #2 - Identify an MMR article |
MMRCS-2 (Name and identify the different types of MMR designs) |
|
5-6 |
Mini-Assignment #1 - Review MMR core features Choosing a mixed methods design |
MMRCS-1 ~ MMRCS-8 |
|
7-8 |
MMR – Core Designs + [assign MMR case study] |
MMRCS-4 (Explain MMR "convergent" design) MMRCS-5 (Explain MMR "explanatory" design) MMRCS-6 (Explain MMR "exploratory" design) |
|
9 |
What can a MMR study distinctively accomplish? Activity #3 – Purpose of mixing assessment |
MMRCS-3 (Identify different purposes of mixing) |
|
10-11 |
MMR Complex Design Application – Cervical Cancer Screening Program (phases I, II, & III) (optional self-study video) Evaluating the Long-Term Impact of a Taiwan Study Abroad Program on Participants’ Career Choice & Professional Development in Healthcare: A Convergent Mixed Methods Study |
MMRCS-1 ~ MMRCS-8
|
|
12 |
Activity #4 – Review assigned MMR case study with team |
MMRCS-7 (Critically analyze an MMR case study) MMRCS-8 (Critically evaluate MMR study quality) |
|
13-14 |
Mini-Assignment #2 – MMR case study [team] report (I) & (II) |
MMRCS-1 ~ MMRCS-8
|
|
15-16 |
MMR end-course reflection & course feedback Activity #5 - Guided reflection on MMR learning |
MMRCS-1 ~ MMRCS-8
|
*MMR Competency Scale (MMRCS): The MMRCS_8 is a condensed version of the original, research-validated 15-item MMRCS [9].
Participants
This study evaluated four course cohorts taught from 2022 to 2025 (N = 65) by the same instructor, who was appointed as a visiting professor to teach the course at two large national universities in northern Taiwan. Students were primarily from public health, nursing, and other health-related disciplines. The course was offered through institutional invitation, with each university formally approving it as a graduate elective. Human subjects approval was obtained from the author’s U.S. institution (IRB ID# STUDY00001672).
Measurements
Three researcher-developed surveys served as the primary data sources: a baseline pre-course survey, a mid-course feedback survey, and an end-course evaluation. Each survey included both quantitative and qualitative components. The baseline survey captured students’ motivation and worries regarding the elective course. The mid-course survey assessed the perceived effectiveness of instructional elements, students’ learning progress, and areas for instructional adjustment. The end-course survey collected reflections on learning outcomes, perceived impact, and overall course experience [8,9].
Quantitative Measures
Quantitative data were collected through anonymous online surveys using 5-point Likert scales. Students rated the helpfulness of core instructional elements-including lectures and Q&A, in-class activities and discussions, assignments and reports, case studies, and article reviews-in supporting their MMR learning (1= not helpful to 5 = very helpful). Overall Student Perception of Instruction (SPI) was also rated on a 5-point scale to capture global impressions of instructional quality.
Qualitative Measures
Qualitative data were obtained from open-ended survey items to capture students’ reflections on their learning experiences. In the mid-course survey, students were asked to identify course elements they found most beneficial and recommended retaining, such as “What are things that you like about this course that you would suggest to keep?” [8]. The end-of-course survey invited students to describe specific experiences that supported the development of MMR knowledge and skills, with examples where possible (e.g., “What were the course experiences that you identify as effective for developing MMR knowledge and skills? Please illustrate with examples if possible.”). To elicit deeper insights into students’ learning processes and challenges, structured reflective prompts included: “It most helped my learning of the content when… because…,” “The approach I took to my own learning that contributed the most for me was… because…,” “The biggest obstacle for me in learning the material was… because…,” and “What would you say about the course and/or instructor if you would like to recommend it to other students?” [9]. Two additional prompts assessed post-course changes and perceived career impact: “After completing this MMR short course, what change do you have on your prior worry or concern? Please briefly explain the change and the key factor that led to it,” and “What impact or change do you think this MMR short course training might have on your research or career? How do you intend to use MMR (or not) in the future?” Collectively, these responses provided rich qualitative data about the instructional strategies and experiences that students found most impactful.
Data Analysis
This study employed a qualitative-dominant mixed methods process evaluation design. Qualitative data were prioritized to capture students’ learning experiences in depth, while quantitative data provided complementary context and triangulation. Mid-course open-ended responses on “things to keep” and end-course reflections on “most helpful strategies” and “effective course experiences” were analyzed using a deductive coding approach based on pre-identified course elements. Text responses were reviewed systematically with analytic memos and annotations to capture student perspectives on these elements [8].
A broader qualitative analysis was conducted for other reflective items from the end-course survey, such as perceived learning obstacles, self-directed learning strategies, and how students would describe or recommend the course to others. These responses were analyzed using theme-based text analysis [19], with the individual student as the unit of analysis. An inductive approach guided the coding process, allowing themes to emerge from the data. Iterative coding cycles were used to identify recurring patterns and refine the thematic structure. Codes were then clustered into broader themes that aligned with the evaluation goals, ensuring both analytic rigor and responsiveness to students expressed experiences [18,19]. The final themes were confirmed through a cross-source synthesis and reviewed by two researchers to enhance credibility.
To integrate qualitative and quantitative results, visual joint displays were developed to facilitate interpretation across data types. Such displays are particularly valuable in mixed methods research for illustrating convergence, divergence, or expansion between data strands. Table 2 links students’ mid-course qualitative feedback on course elements they recommended retaining with corresponding quantitative Likert-scale ratings and integrates themes from end-course responses, such as “most helpful teaching strategies” and “effective course experiences,” with ratings of related instructional components. Table 3 presents student-identified learning strategies, with accompanying frequency counts to provide a comprehensive view of engagement and perceived impact. Finally, Table 4 summarizes post-course reflections on changes in students’ initial worries and the perceived impact on their future MMR research trajectories, combining code frequencies with illustrative quotes to highlight key themes.
Descriptive statistics were computed for Likert-scale items assessing the perceived helpfulness of course elements across mid- and end-course surveys, as well as overall Student Perception of Instruction (SPI). These quantitative results contextualized the dominant qualitative themes and highlighted areas of alignment across student responses.
Table 2: Teaching Approaches to Keep and Most Helpful Course Experiences.
|
Teaching Approaches to keep* |
Most Helpful Course Experience** |
||||
|
Course Elements Mean (SD) |
Code Frequency (%) |
Sample quotes |
Code Frequency (%) |
Course Elements Mean (SD) |
Sample quotes |
|
Class lectures & Q/A 4.34 (0.626) |
5 (15%) |
[The lecture is very clear and concise. (ID#4,50, 53)] [Very helpful, after the class, I have a much better understanding of MMR and know the difference between each type. (ID#10, 43)] |
In-class activities & discussions 12 (26%) |
Class lectures & Q/A 4.64 (0.554)
In-class activities & discussions 4.63 (0.620) |
[When more philosophy behind MMR was discussed because it help us to know the limitations and advantages of MMR. (ID#3, 10, 12, 40, 41)] [Doing small exercises or group discussion because I can test if I really know the content or find out what I haven't understood clearly. (ID#23, 30, 39, 43, 44)] [When we discussed the different purposes of MMR because it is the beginning of our own study proposal. (ID#20)] |
|
In-class activities & discussions 4.24 (0.918) |
10 (29%) |
[Through these discussions, I observed the collection of ideas among classmates, which further solidified the concepts. This part was highly recommended to keep. (ID#47, 62)] [Group discussion. It’s fun and improves English, hearing diverse opinions. (ID#9, 18, 19, 29, 30, 51, 52, 59)] |
|||
|
Class assignments & reports 4.05 (0.931) |
1 (3%) |
[Assignment helped me quickly catch the key point about the characteristics of each MMR design instead of reading 7 papers.... (ID#17)] |
Outside class mini assignments 6 (13%) |
Class assignments & reports 4.48 (0.687) |
[When doing the group case study report, I can learn different thoughts from my partner. (ID#4)] [When I did the practice of the assignment because I could try to realize the application of MMR. (ID#16, 32, 34, 50, 51)] |
|
MMR case studies 4.18 (0.859) |
8 (24%) |
[A real case study is quite helpful to understand the concept and utilization of MMR. (ID#8, 22, 27, 35, 45)] [Case examples and references can help interpret MMR correctly. (ID#31, 32)] |
MMR course reflection report 13 (28%) |
MMR case studies 4.61 (0.593)
MMR article review 4.50 (0.632) |
[More case studies could inspire my thinking. I am looking forward to the next MMR class. (ID#7, 38)] [When studying the cases, it could make me realize the core features of MMR and think about how to apply it. (ID#2, 8, 22, 26, 33, 37, 49)] [We critically evaluated articles ... it helped me apply what we learned in class and showed me that in the real world, how effectively MMR methods sections are written varies. (ID#13, 31, 45)] |
|
MMR article review 4.26 (0.767) |
3 (9%) |
[Find articles to discuss and determine which type of mixed research it is. (ID#21, 38) Continued examples from real scientific literature and critical evaluation of the written quality of their methods sections. (ID#15)] |
|||
|
Instructor-design-environ-pace |
7 (20%) |
[[The instructor] always asks if there are comments or questions (ID#12)] [Very clear PowerPoint slides, good class engagement. (ID#24) The overall is awesome. (ID#1, 5, 7, 14)] |
Instructor lecture, guidance, & project consultations 15 (33%) |
|
[Not over pressure students but mostly focused on the participation and the QA. (ID#5, 14, 35)] [When [the instructor] explained in real examples, it helped me better understand how and when to use it in a realistic way. (ID#1, 17, 21, 25, 36)] [Give me some brief introductory reading prior to classes. (ID#9, 19)] |
|
Total |
34 (100%) |
|
46 (100%) |
|
|
Note. Data source: *Mid-course feedback; **End-Course Survey.
Table 3: Approaches the Student Took to Learn.
|
Themes |
Code Freq. (%) |
Sample Quotes |
|
Discussion-Based Learning |
11 (34%) |
· Having discussions with my classmates helps me learn more. (ID#2, 4, 8, 17, 36, 43) · To discuss the articles because I can review what I've learned and learn from other classmates' perspectives. (ID#35, 37, 41, 44) |
|
Note-Taking, Review, and Individual Study |
7 (22%) |
· Referencing my notes from class and seeing how they apply to articles/case studies ... that practice helped cement the lecture material in my head (ID#3, 13, 38, 39) · Class participation and review of notes, because I found the content to be straightforward and accessible. (ID#22, 28, 45) |
|
Resource Exploration, Reading, and Application |
12 (38%) |
· It is conducting complicated studies and enhancing credibility in the study. (ID#6, 7, 27, 33) · Review articles using MMR in my research interests because I can easily imagine the benefits of using MMR. (ID#12, 19) · Reading every paper provided because practice is a good way to learning (learning from doing) (ID#20, 21, 49) |
|
Overcome Language Barriers |
2 (6%) |
· Translation, because it helped me understand better. (ID#31) |
|
Total |
32 (100%) |
|
Note. Data source: End-course survey.
Table 4: Changes in Worry Post-Course and Perceived Impact on Research and Career.
|
Subsequent Changes of Worry or Concern Following Course Completion |
Impact on Research or Career Path |
||||
|
Themes |
Code Freq. (%) |
Student Sample Quotes |
Themes |
Code Freq. (%) |
Student Sample Quotes |
|
Growth in Understanding of MMR |
14 (31%) |
Have a better understanding of the beauty of MMR. (ID#2, 3, 41, 43, 49)
I learned about the key information related to this methodology, and the case study really helped to verify what was learned and absorbed. (ID#8, 14, 30, 35, 36)
I can read the MMR study more clearly and explain the text more easily. (ID#7, 10, 39) |
Increased MMR confidence |
19 (36%) |
Before I attended this class, I was just thinking of applying MMR in my research, and after the class, I think I am somewhat confident to use MMR in my future studies. (ID#6, 8, 9, 13, 29, 44, 45, 47)
I feel much more confident in my ability to critique and explain MMR. (ID#6, 9, 10, 12, 13, 21, 41, 44, 45, 47, 48) |
|
Language and Communication Support |
8 (18%) |
When I express myself in my poor English, no one would laugh or show any impatient expression... my classmates or [the instructor] will wait, encourage, and give some comments. That really helps me go through my worries. (ID#5, 17) I can understand what the assignment was and discuss with others. (ID#1, 16, 21, 24) |
Intent to apply MMR |
18 (34%) |
This course inspired me to learn more about MMR. I would like to use MMR in my dissertation. (ID#3, 10, 23, 31, 32, 34, 38, 41, 51)
I would try to do an MMR study in the future, and encourage others as well because it is interesting, useful and challenging. (ID#1, 7, 17, 12, 24-26). |
|
Value of Instructional Approach and Instructor Support |
17 (38%) |
Although the teacher used English in class, the speed and the way of explanation were not as difficult to understand as I imagined, so after the first hour of class, my prior worries were eliminated. (ID#11, 13, 19, 25, 26, 29, 37)
After listening to students' feedback, the teacher proceeded with the class in a slow way and repeated key terms in Chinese. (ID#27, 40)
The teacher clearly clarifies the different types of design and purpose so that we can clearly understand and apply them to our own study. (ID#18, 20, 22, 23, 38, 44, 48, 51) |
Conditional or uncertain use of MMR |
6 (11%) |
Knowing MMR explores my vision of research. Can use both Qn and Ql ways to understand things better, but it depends on the resource’s availability. (ID#11, 19, 20)
I won’t use MMR for my dissertation, but I would think about finding a chance to do MMR.” (ID#16, 30, 35, 42, 46, 49) |
|
Desire for Continued Learning and Application |
6 (13%) |
I might need to have more classes about qualitative methods if I want to use mixed methods to conduct studies. (ID#4, 6)
Realize the concepts of MMR and look forward to applying them in future studies correctly. (ID#31, 34, 47) |
Advocate MMR to transform one’s field |
10 (19%) |
MMR solves many of the issues I've had choosing between quantitative and qualitative... it’s heartening to see a field dedicated to this. (ID#13, 20, 26, 27, 28, 36, 37, 39, 50, 52)
The MMR thought process can be applied to how I approach various issues in my daily life. (ID#27) |
|
Total |
45 (100%) |
|
|
53 (100%) |
|
Note. Data source: End-course survey.
Results
Teaching Approaches to Keep and Most Helpful Course Experiences
Quantitative survey results indicated that all core course elements were rated as helpful to very helpful in supporting student learning, with mean ratings increasing from 4.05-4.34 at mid-course to 4.48-4.64 at the end of the course. This upward trajectory reflects students’ growing appreciation of the course’s effectiveness.
Qualitative analysis of 34 mid-course and 46 end-course responses revealed that students especially valued interactive and applied learning strategies. Mid-course feedback most frequently highlighted in-class activities and discussions (29%), followed by MMR case studies (24%) and the emergent “instructor factor” (20%), which encompassed guidance, course pacing, and supportive learning environment. End-course reflections consolidated into four integrated themes: instructor lectures, guidance, and project consultations (33%); the MMR course reflection report, which combined article reviews and case studies (28%); in-class activities and discussions (26%); and outside-class mini assignments (13%). Students emphasized that instructor explanations using real-world examples demystified complex MMR concepts: “When [the instructor] explained in real examples, it helped me better understand how and when to use it in a realistic way.” Reflective assignments deepened engagement with the literature: “We critically evaluated articles… it helped me apply what we learned in class and showed me that in the real world, how effectively MMR methods sections are written varies.” Students also valued peer discussion and hands-on activities for reinforcing understanding: “Doing small exercises or group discussion… I can test if I really know the content or find out what I haven't understood clearly.”
Across both surveys, students highlighted the synergy of structured instruction (e.g., instructor lectures and guidance) (33%), interactive learning (e.g. in-class discussions and peer learning) (28%), and authentic application (e.g., mini assignments and reflective work) (26%) as key to mastering foundational MMR skills. These findings highlight the effectiveness of a multimodal instructional approach in teaching MMR (Table 2).
Biggest Challenges and Learning Approaches
Thematic analysis of 41 end-course reflections identified three primary challenges. The most frequently reported challenge (37%) was English-language comprehension, as the course was part of the institution’s English as a Medium of Instruction (EMI) initiative. Students noted the need for extra time to process content, with one explaining, “The biggest obstacle was that I needed some time to process the class information because I am not a native speaker.” Limited foundational knowledge in either qualitative or quantitative methods was the second challenge (27%), which hindered students’ ability to fully grasp MMR integration:“MMR is a new dimension… many of the terminologies of the methods are new.” Time constraints (14%) were a smaller but notable concern, reflecting the compressed nature of the two-day format. Some students wished for more time to read assigned papers or extend small-group discussions.
Analysis of 32 responses about learning approaches revealed four strategies. The most common was resource exploration, reading, and application (38%), as students actively engaged with assigned materials: “Reading every paper provided because practice is a good way to learn.” Discussion-based learning (34%) allowed peer exchange to reinforce understanding: “To discuss the articles because I can review what I’ve learned and learn from other classmates’ perspectives.” Note-taking and individual review (22%) supported content retention, while a small subset (6%) used translation tools to overcome language barriers: “Translation… helped me understand better.” These findings illustrate both the challenges of an EMI short-course format and the adaptive strategies students employed to succeed (Table 3).
Motivations for Taking the MMR Short Course
Baseline survey results revealed that students primarily enrolled in the MMR short course for professional and academic development rather than to fulfill degree requirements. The top-ranked reason, selected by 41.1% of students, was to better prepare for future research and career opportunities using MMR, followed by the desire to build methodological competencies for graduate studies (35.7%). Fulfilling a program requirement was rarely the primary motivator (7.1%) and was most often ranked last (44.6%), suggesting intrinsic interest and perceived professional relevance drove enrollment. Excitement about taking a class with the instructor appeared most often in lower ranks (44.6% as third and 33.9% as fourth), reflecting the influence of reputation and peer recommendations, despite students having no prior experience with the instructor. One student noted they were “curious to experience the teaching style others had recommended.” Overall, these results indicate that students were primarily motivated by skill acquisition and career relevance, with instructor credibility contributing to their decision to enroll.
Changes in Worry Post-Course and Perceived Impact on Research and Career
Analysis of 45 end-course reflections identified four themes regarding reduced worry following course completion (Table 4). The most common (38%) was the value of instructional approach and instructor support, particularly for non-native English speakers. One student remarked, “Although the teacher used English in class, the speed and way of explanation were easier to understand than I expected, so after the first hour, my prior worries were eliminated.” Growth in understanding of MMR (31%) reflected improved comprehension and confidence, often reinforced through case studies: “The case study really helped to verify what was learned and absorbed.” Language and communication support (18%) underscored the psychologically safe classroom climate, with students noting that peers and the instructor were patient and encouraging. Finally, 13% expressed a desire for continued learning, recognizing the need for deeper study of qualitative methods to advance their MMR skills.
Thematic analysis of 53 responses highlighted four keyways the course influenced students’ research and career trajectories (Table 4). The most frequently reported impact was increased confidence in MMR (36%), with one student reflecting, “Before I attend this class I was thinking to apply MMR in my research, and after class, I think I am somehow confident to use MMR in my future studies.” A closely related theme was intent to apply MMR in future research (34%), including planned integration into dissertations or teaching: “I would like to use MMR for my PhD dissertation regarding bedtime procrastination.” Nineteen percent of students described MMR as transformative for their field, enabling interdisciplinary thinking and problem-solving: “MMR solves many of the issues I've had choosing between quantitative and qualitative.” A smaller group (11%) expressed conditional or uncertain plans to use MMR, citing fit and resource limitations: “I won’t use MMR for my dissertation, but I would think about finding a chance to do MMR for a topic I’m curious about.”
Together, these results indicate that the course reduced student anxiety or worry, strengthened methodological confidence, and fostered intentions to apply MMR in both immediate and long-term research trajectories.
Student Recommendations and Reflections
Thematic analysis of 51 responses to the prompt, “What would you say about the course and/or instructor if you would like to recommend it to other students?” revealed three primary themes: the value of MMR (51%), the instructor factor (31%), and course design (18%).
The most frequently cited theme-MMR value-reflected students’ appreciation for the course’s practical relevance, bilingual delivery, and rarity within the local academic context. One student described it as “intensive, time-saving, bilingual-a rare opportunity in Taiwan,” underscoring its accessibility and usefulness for research training. The instructor factor highlighted the instructor’s clarity, supportiveness, and ability to simplify complex concepts while fostering a welcoming environment. Students remarked, “[The instructor] guides to core process, provides teamwork support, and speaks Chinese,” and “[The instructor] explains in easy ways [for understanding].” The course design theme emphasized the class’s structured, coherent format, which provided a strong introduction to MMR and a foundation for future research: “This is a good course to understand MMR, useful for future research.” Collectively, these reflections highlight that students valued the integration of relevant content, thoughtful design, and strong instructional support in meeting their academic and professional goals.
Student Perception of Instruction (SPI)
Across four course cohorts (2022–2025), students consistently rated instructional quality very highly, with a mean Student Perception of Instruction (SPI) score of 4.69 (SD = 0.51) out of 5 based on 49 end-course survey responses. These consistently strong ratings indicate stable perceptions of instructional excellence across repeated course offerings. Open-ended comments further illustrated students’ appreciation for the course’s supportive and engaging learning environment. Many expressed enjoyment and gratitude, contrasting the experience with traditional high-pressure classroom settings: “I really appreciate the opportunity to learn the lesson, but most importantly, to enjoy the class like this. Very different from the military style classes we used to have. Little stress but lots of gain” (ID#1, 5, 9, 59, 62). Students also praised the instructor’s patience and encouragement when navigating English-language participation: “Even though we often feel nervous or short of words when speaking English, the professor is still patient enough to let us practice” (ID#12, 24, 25, 27, 56). Another summarized, “The course is so excellent and wonderful that I will encourage the other students to join!” (ID#17, 34). Together, these reflections reinforce the instructor’s pivotal role in creating a student-centered, multimodal learning environment and affirm the course’s success in providing accessible, confidence-building MMR training.
Discussion
This mixed methods process evaluation of a two-day intensive MMR short course provides key insights into teaching complex methodological content in condensed timeframes. The convergence of quantitative and qualitative data underscores the effectiveness of an integrated, multimodal instructional design. All core course components received high student ratings at end-course (means 4.48–4.64), and qualitative feedback highlighted reduced worry, improved understanding, and strong engagement. The combination of bilingual instruction, clear pacing, and a supportive classroom environment addressed language-related anxiety, while interactive lectures, case studies, and mini-assignments provided immediate, hands-on practice. These findings align with prior research emphasizing active, multimodal approaches to methods education [5,8,9].
Pedagogical Effectiveness and Learning Gains
Student reflections revealed that the course successfully shifted perceptions of MMR from a simple combination of qualitative and quantitative methods to a recognition of its integrative methodological complexity. Key instructional elements-interactive activities, reflection reports, and mini-assignments-were credited with reinforcing understanding, while peer discussion and in-class engagement were particularly valued, supporting calls for active learning in research methods education [8,14,20]. Instructor presence emerged as central to student success; 33% of end-course comments cited guidance, real-world examples, and personalized support as critical to learning. These findings echo research on instructor impact in intensive formats [21] and align with situated learning theory, which emphasizes contextualized, applied learning [22]. Bilingual delivery and culturally responsive pedagogy further contributed to a psychologically safe and inclusive learning environment [23].
Challenges and Adaptive Strategies
Despite positive outcomes, students reported challenges including English comprehension (37%) and limited foundational knowledge in either qualitative or quantitative methods (27%). These align with prior findings on language-related cognitive load [24] and disciplinary diversity in mixed methods courses [6]. Students responded with adaptive strategies such as translation tools, independent reading, and peer discussion, reflecting social constructivist learning principles that emphasize active knowledge construction through interaction [25].
Motivation, Impact, and Course Replicability
Baseline data indicated that most students enrolled for professional and research development rather than degree requirements, highlighting intrinsic motivation as a driver of engagement [26,27]. End-course reflections confirmed increased confidence, intent to apply MMR in future work, and recognition of its value for interdisciplinary research. Consistently high Student Perception of Instruction (SPI) scores (mean = 4.69) across cohorts support the replicability and sustainability of the course model. Integrated assessments-including mini-assignments, case analyses, and article reviews-appeared especially effective, consistent with competency-based approaches emphasizing applied skill development [8-11,18].
Value of Mixed Methods Process Evaluation
The use of a mixed methods process evaluation provided a nuanced understanding of both course effectiveness and learning mechanisms [8,10,11]. Joint display analyses highlighted where quantitative and qualitative findings converged-such as the impact of instructor guidance-and revealed emergent insights, including language challenges that may be overlooked in quantitative-only evaluations. This approach reflects Moore, et al [13] calls to move beyond outcomes to illuminate how and why educational interventions succeed, and it models MMR as a valuable framework for teaching and evaluating complex pedagogical innovations [28,29].
Limitations and Future Directions
The study was conducted in Northern Taiwan, which may limit generalizability. Success was likely influenced by the bilingual instructor, and replication by English-only instructors may yield different outcomes. Voluntary enrolment may also have attracted highly motivated students, potentially inflating positive evaluations. Future research should explore the influence of instructor characteristics, native-language versus English delivery, and the scalability of short-course models. Comparative studies of in-person, hybrid, and online formats would further inform optimal design for complex methodological content.
Conclusion
This study demonstrates that a thoughtfully designed, multimodal MMR short course can effectively build graduate students’ methodological competencies within a compressed timeframe. Key elements-active learning, reflective practice, culturally responsive pedagogy, and strong instructor engagement-contributed to reduced apprehension, increased confidence, and sustained positive perceptions across cohorts. The mixed methods evaluation not only confirmed the course’s effectiveness but also illustrated the value of MMR for capturing the mechanisms that drive educational impact. As demand for methodological pluralism grows, short, well-structured MMR courses offer a scalable model for fostering research competence and advancing methods education in diverse academic contexts.
Acknowledgement
Special thanks to Professor Shu-Sen Chang, Director of the Institute of Health Behaviors and Community Sciences, College of Public Health at National Taiwan University (NTU), for his strong support in the development and implementation of the mixed methods research short course. Warm appreciation also goes to Dr. Po-Yao Huang for co-facilitating the course delivery, coordinating academic affairs, and assisting with student recruitment and course promotion at NTU. Sincere gratitude is extended to Professor Yu-Ching Chou, Director of the Institute of Public Health at National Defense Medical University (NDMU), for his enthusiastic support in extending the course to NDMU students and for his significant efforts in promoting and endorsing its offering. This course was made possible through EMI funding support from NTU, along with departmental and university-level support from both NTU and NDMU. Finally, heartfelt thanks to all participating students for their active engagement and contributions in creating a warm, joyful, meaningful, and collaborative learning environment.
References
- Creswell JW, Clark VLP. Designing and conducting mixed methods research (3rd Edition). Sage Publications. 2018.
- Zhou Y, Zhou Y, Machtmes K. Mixed methods integration strategies used in education: A systematic review. Int J Research & Method in Education. 2024; 47: 158-175.
- Silverman SL. A mixed-methods research design to advance inclusive and equitable teaching. Innovative Higher Education. 2024; 49: 1105-1125.
- Gierus B, Du T, Maduforo AN, Gilbert B, Koh K. Prevalence and quality of mixed methods research in educational subdisciplines: A systematic review. SAGE Open. 2025; 15.
- Younas A, Durante A, Fabregues S, Escalante-Barrios EL. Strategies for educators to teach mixed methods research: A discussion. The Qualitative Report. 2024; 29: 831-845.
- Sammons P, Davis S. Mixed methods approaches in educational research. In Oxford Research Encyclopedia of Education. Oxford University Press. 2017.
- Hou S-I. My serendipity journey to mixed methods research (MMR) dissertation, teaching & learning in the U.S.A. Annals of Mixed Methods Research. 2022a; 1: 61-67.
- Hou S-I. A mixed methods process evaluation of an integrated course design on teaching mixed methods research. Int J Scholarship of Teaching and Learning. 2021; 15.
- Hou S-I. A mixed methods outcome evaluation on mixed methods research competencies. Int J Teaching and Learning in Higher Education. 2025; 35: 20-35.
- Hou S-I. Advancing a competency-based mixed methods tool to assess a course-based service-learning model integrating real-world experience. Canadian J Program Evaluation. 2022b; 36: 391-407.
- Hou S-I. A mixed methods evaluation on teaching evaluation: Innovative course-based service-learning model on program evaluation competencies. J Scholarship of Teaching and Learning. 2022c; 22: 95-112.
- Marchal B, Van Belle S, Van Olmen J, Hoeree T, Kegels G. Is realist evaluation keeping its promise? A literature review of methodological practice in health systems research. Evaluation. 2012; 18: 192-212.
- Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015; 350: h1258.
- Kakai H. The mixed methods treasure hunt: Reflecting on the legacy of Dr. Michael D. Fetters in teaching mixed methods research. J Mixed Methods Research. 2024; 18: 245-261.
- Hou S-I. Beyond borders: Dr. Fetters’ global legacy in mixed methods. Annals of Mixed Methods Research. 2024; 2: 235-240.
- Council on Education for Public Health. Accreditation criteria: Schools of public health and public health programs. 2016.
- Council on Education for Public Health. Accreditation criteria for schools of public health and public health programs: Criterion D2-MPH foundational competencies. 2024.
- Guetterman TC, Babchuk WA, Howell Smith MC, Stevens J. Contemporary approaches for mixed methods research in education: A practical guide. Int J Multiple Research Approaches. 2019; 11: 2-20.
- Creswell JW. 30 essential skills for the qualitative researcher. Sage Publications. 2016.
- Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H, et al. Active learning increases student performance in science, engineering, and mathematics. Pyschological and Cognitive Sciences. 2014; 111: 8410-8415.
- Garrison DR, Kanuka H. Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education. 2004; 7: 95-105.
- Lave J, Wenger E. Situated learning: Legitimate peripheral participation. Cambridge University Press. 1991.
- Gay G. Culturally responsive teaching: Theory, research, and practice (3rd Edition). Teachers College Press. 2018.
- Sawir E. Language difficulties of international students in Australia: The effects of prior learning experience. Int Education J. 2005; 6: 567-580.
- Vygotsky LS. Mind in society: The development of higher psychological processes. Harvard University Press. 1978.
- Earley MA. A synthesis of the literature on research methods education. Teaching in Higher Education. 2014; 19: 242-253.
- Clark VLP, Ivankova NV. Mixed methods research: A guide to the field. Sage Publications. 2016.
- Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-Principles and practices. Health Services Research. 2013; 48: 2134-2156.
- Guetterman TC. What distinguishes a novice from an expert mixed methods researcher? Quality and Quantity. 2017; 51: 377-398.
- Hou S-I. Mixed-methods evaluation: A Chinese women cancer screening program. UCF Press books. 2020.
- Hou S-I, Fetters MD. Mixed methods in public health research in Taiwan: Using visual diagrams to communicate complex design procedures. Health Care for Women Int. 2019; 40: 515-526.