- © 2010 American Dental Education Association
A Systematic Review of Computer-Assisted Learning in Endodontics Education
- Thikriat S. Al-Jewair, B.D.S., M.B.A., M.Sc., F.R.C.D.(C),
- Akram F. Qutub, B.D.S., Ph.D., F.R.C.D.(C),
- Gevik Malkhassian, D.D.S., M.Sc., F.R.C.D.(C) and
- Laura J. Dempster, B.Sc.D., M.Sc., Ph.D.
- Direct correspondence and requests for reprints to Dr. Thikriat S. Al-Jewair, Department of Orthodontics, School of Dental Medicine, University at Buffalo, 140 Squire Hall, Buffalo, NY 14214; .
- Received October 9, 2009.
- Accepted January 12, 2010.
Results of the efficacy and time efficiency of computer-assisted learning (CAL) in endodontics education are mixed in the literature. The objectives of this study were to compare the efficacy and time efficiency of CAL with traditional learning methods or no instruction. The search strategy included electronic and manual searches of randomized controlled trials (RCTs) completed in English up to June 2009. The intervention comprised any method of CAL, while the control group consisted of all traditional methods of instruction including no further instructions. Various outcome measures of CAL efficacy were considered and were categorized using Kirkpatrick’s four-level model of evaluation: reaction, learning, behavior, results, with the addition of return on investment as a fifth level. The time efficiency of CAL was measured by the time spent on the learning material and the number of cases covered in a unit period. Seven RCTs met the inclusion criteria. Overall, students’ attitudes were varied towards CAL. Results from the knowledge gain outcome were mixed. No conclusions can be made about students’ performance on clinical procedures or cost-effectiveness of CAL. Better time efficiency was achieved using CAL compared to traditional methods. CAL is as efficacious as traditional methods in improving knowledge. There is some evidence to suggest that CAL is time efficient compared to traditional methods. Overall, the number of studies included in this review was small, thus warranting the need for more studies in this area and the exploration of various CAL techniques.
Computer-assisted learning (CAL) is increasingly used in dental curricula to support the traditional teaching and learning of dental students in various subject areas. CAL is being employed in various forms, such as computer or web-based tutorials or discussion groups, computerized patient simulations, and virtual reality-based simulations.1
CAL offers many advantages to dental students, educators, and academic institutions. Students benefit by individualizing the learning process, allowing them to work at their own pace and to repeat the learning program as much as required.2,3 For educators, CAL reduces direct instructor/student contact time, thereby enabling faculty members to have more time to perform other academic activities.4 Academic institutions benefit from self-instruction using CAL as it helps alleviate the problem of staff shortages. However, the start-up costs of CAL programs are immense in terms of monetary resources, training of educators and students, and the time necessary to develop and structure such programs.4
A previous meta-analysis of thirty-four comparative studies in dentistry reported small to moderate effect of CAL over conventional instruction methods.5 The investigators also found a 75 percent reduction in instruction time compared to traditional methods. Pooled evidence from a recent meta-analysis6 on CAL efficacy in orthodontic education revealed that CAL is as efficacious as traditional methods and adds a statistically significant knowledge gain among predoctoral students ranging from 3.8 to 9.8 percent. It also concluded that students’ attitudes were positive towards CAL.
The practice of endodontics requires problem-solving knowledge and skills in order to derive an accurate diagnosis, treatment plan, and patient prognosis.7,8 To reduce these challenges, a number of CAL programs have been developed, evaluated, and utilized to improve student learning and clinical outcomes in endodontics. Fouad and Burleson9 evaluated the use of an endodontic computer simulation program in teaching diagnosis. They found a significant gain in knowledge in the simulation group. On the other hand, Plasschaert et al.10 found no significant difference in knowledge after the use of interactive multimedia simulated cases as compared to traditional text-based cases.
In an attempt to evaluate whether the use of CAL is efficacious in endodontics education, this systematic review was undertaken to 1) compare the efficacy of CAL with traditional methods of learning in predoctoral and postdoctoral endodontics education or with no instruction and 2) compare the time efficiency of CAL programs versus traditional methods or no instruction.
A comprehensive search was executed of English articles completed up to June 2009 using the following databases: Ovid MEDLINE, Ovid OLDMEDLINE, EMBASE, Database of Abstracts of Reviews of Effectiveness (DARE), Educational Resources Information Center (ERIC), International Pharmaceutical Abstracts (IPA), Cumulative Index to Nursing and Allied Health Literature (CINAHL), HealthSTAR/Ovid HealthSTAR, Cochrane Controlled Trials Register, Cochrane Database of Systematic Reviews, Health and Psychosocial Instruments, Library and Information Science Abstracts (LISA), PubMed, and the worldwide web using Google Scholar. The search included categories of computer-aided instruction and endodontics and combinations of key words. Manual searches of reference lists of relevant articles and of theses and dissertations were also performed.
PICO (Population, Intervention, Comparator, and Outcome) criteria were used to guide the inclusion or exclusion of studies in this review. Only original randomized controlled trials (RCTs) were included. The study population was limited to predoctoral dental students taking an endodontics course, postgraduate endodontics residents, and endodontics staff. The intervention comprised any method of CAL, including computer-based patient simulation, virtual reality simulation, multimedia learning programs, hypertext systems, and a CAL software program for teaching diagnosis and treatment planning in endodontics. The control group consisted of all traditional methods of instruction: live lectures, laboratories, seminars, tutorials, text- or note-based learning, slide-tape methods (through the use of slides and audiotapes with no computer aid), and no instruction.
Various outcome measures of CAL efficacy were considered and categorized using Kirkpatrick’s four-level model of evaluation of learning and training in the human resources field: reaction, learning, behavior, and results,11 with the addition of a fifth level, return on investment (ROI), that was added by Phillips12 (Table 1⇓). Level 1 (reaction) measured participant reaction to the learning intervention: Did participants like the intervention? Was it relevant to their course of study? Level 2 (learning) addressed learning acquisition that occurred after the intervention or post-test, as compared to pre-test: Was there an improvement in knowledge? Level 3 (behavior) evaluated whether there was a change in behavior as a result of the intervention received: Was there a measurable change in activity and performance in the learner’s environment? Level 4 (results) evaluated the overall effect of the intervention on the institution: Was there an improvement in quality of care and productivity level? The final level (ROI) addressed the monetary value of the results and their costs: Did the monetary value of the intervention exceed its costs? The time efficiency of CAL was measured by the time spent on the learning material and its correlation to the students’ test scores or the number of cases completed in a unit period.
Data Extraction and Quality Assessment
Two independent reviewers (TJ and AQ) assessed the studies by inspecting titles, abstracts, and full articles for validation and quality assessment. The guidelines of the Canadian Task Force on Preventive Health Care13 were used to assess the level of evidence and recommendations on grades for specific clinical preventive actions in retrieved articles. The articles were then appraised and scored using the “Checklist for Appraising Evidence in Health Care for Studies of Efficacy of Therapy or Prevention” adapted by Dr. James Leake14,15 (Table 2⇓). The checklist consists of seventeen items and addresses study ethics, design and methodology, validity of results, and their applicability. The maximum possible score was 17, with a minimum of 9 needed for inclusion in this review. This score was chosen to reflect the minimum number of checklist questions that define a good quality RCT. Studies were deemed high quality if they had objective pre- and post-intervention assessment, homogeneous groups of participants, random allocation, blinding of outcome assessors (blinding of participants or learning module instructors was not applicable), controlling for confounders, “intent to treat” analysis, and completeness of follow-up of 80 percent or more. Percentage agreement between reviewers was assessed with agreement level of 99 percent and 96 percent achieved at the title and abstract stage, respectively. Differences were resolved through discussion.
Table 3⇓ outlines the search strategy that initially extracted seventy-one relevant studies. Duplicates were removed, after which titles, abstracts, and full articles were reviewed. Two additional studies were then eliminated as they were not randomized control trials.7,16 In total, seven articles met the inclusion criteria for this systematic review.
The studies evaluated the efficacy of CAL in various endodontics areas: history-taking and clinical examination,9,10,17 diagnosis (including etiology, signs and symptoms, diagnostic categories, and diagnostic tests used in endodontics),8–10,17,18 treatment planning,8,10,17 access cavity preparation,19 and clinical diagnostic and treatment errors.20
Homogeneous groups of predoctoral dental students taking endodontics courses participated. Homogeneity of student groups was attained in all studies but one19 by dividing students according to their cumulative dental school grade point average (GPA),8,17,20 pre-test scores,9,10,18,20 sophomore preclinical endodontic laboratory GPA, and junior year endodontic course GPA.20 All studies reported reliability testing of their test instruments.
The sample sizes in the studies ranged from twenty-eight to 105 students. The intervention methods included CAL with a hypertext system,19 computer simulation programs,9,20 and computer media programs.8–10,17,18 The control methods included traditional lectures,8 laboratory tutorials,19 text-based cases,10 self-teaching booklets,18 slide-tape methods,17 or no instructions.9,20 Five of the studies were conducted in the United States, and the other two were from Canada and Iran. Characteristics of included studies are presented in Table 4⇓ (pp. 606–9).
Efficacy of CAL
Four outcomes measuring the efficacy of CAL were examined in the studies: attitudes of students toward CAL (reaction), knowledge gain (learning), performance on clinical procedures (behavior), and cost identification (ROI).
Attitudes toward CAL.
This qualitative outcome was evaluated in six studies using questionnaires administered to students. Two studies18,19 found no significant difference in attitude between the two methods. One study10 reported significantly positive attitudes among the CAL group (p<0.05). Three other studies9,17,20 reported varied responses. An overall varied experience towards CAL was noted by participants in the majority of the studies. Quantitative comparison of results was difficult to make due to lack of reporting of results for both methods of learning.
The quantity of knowledge gain was investigated in six studies using multiple choice questions,8,9,18,19 written tests,10 and essays.17 Results varied, with no significant differences reported in three studies10,18,19 and a significant gain in knowledge found in one study.9 In that study, Fouad and Burleson9 compared an endodontics diagnosis computer simulation program to seminars and control (no instruction) and found that the simulation group improved significantly compared to the seminar (p<0.05) and control groups (p=0.0024). Mixed results were reported in the two other studies.8,17 Mendel and Scheetz8 compared the use of a computer-assisted self-evaluation module (CASE) with seminars and lectures. The students were divided into four groups: CASE, seminar, CASE and seminar, and lectures. Results showed a significant knowledge gain between pre- and post-intervention tests in the four groups (p<0.05), but failed to identify significant differences between the intervention methods although there was a gain in knowledge of more than 10 percent in endodontics diagnosis and treatment planning among the CASE and seminar groups compared to the seminar and lecture groups. In the sixth study, Mullaney et al.17 compared computer-assisted presentations to slide-tape methods on three phases (2, 3, and 4). The teaching included endodontics test selection and data gathering, diagnosis, and treatment planning. Mixed results were reported, with phases 3 and 4 showing significant improvement in knowledge in test selection only, and phase 2 results not considered due to technical problems with the computer program.
Meta-analysis was not utilized for the studies due to great variability in methodology and reporting such as different modules of learning and lack of post-intervention testing9,10,18,19 and missing standard deviation calculations.8,17 The knowledge gain outcome results showed that CAL is as efficacious as traditional learning methods in terms of knowledge gain; however, the quality of the knowledge gained cannot be evaluated since the test instruments were not presented.
Performance in clinical procedures.
Only one study20 examined the performance on clinical procedures in terms of the percentage reduction in errors made in the clinic. Seven study groups that received different interventions were compared to a control group that received no instructions. They reported significant reduction in diagnostic errors (chief complaint, visual exam, electronic pulp testing, periapical diagnosis, and required films) between the study and control groups (p<0.001), but not in clinical treatment errors (access cavity, foreman integrity, root canal filling to correct length, and condensation) (p=0.09). Meta-analysis was not conducted due to limited evidence on this outcome.
Only two studies reported on costs. Plasschaert et al.10 stated that the cost of producing the multimedia CD-interactive program (PAINDENT) was close to the annual salary of an assistant professor. Sandoval et al.20 calculated the human resources involved in developing the different CAL, slide-tape, and latent image simulation programs used in their study and found that the computer-assisted video interactive simulation (CAVI) required the highest human resource hours (222 hours) and numbers (n=7) compared with other methods. None of the studies conducted cost-effectiveness analyses; therefore, no inferences can be made about this outcome.
Time Efficiency of CAL
Of the studies selected for inclusion in this systematic review, only two reported on time efficiency. Fouad and Burleson9 objectively counted the number of cases reviewed during a unit period. Of the three cohort classes, students in the computer simulation group covered significantly more cases (mean±standard deviation [SD]; 8.3±2.5) than students in the seminar groups (5.1±1.1; p<0.0001), albeit students gave varied responses on whether CAL was more efficient than seminars. In another study,10 efficiency was evaluated by students’ reporting of time spent on two learning methods (multimedia program or text-based). Students in the multimedia group reported spending significantly less time on average (ninety-eight minutes) compared to students in the control group (131 minutes). Although the studies measured the outcome in different ways, they both identified better time efficiency using CAL programs.
This systematic review assessed the efficacy of CAL in endodontics education as compared to traditional methods. Seven studies met the inclusion criteria. Of the different outcomes evaluated in the studies, no one study considered all possible outcomes.
Six studies reported results on knowledge gain, but did not evaluate the translation of this knowledge into clinical practice, which is arguably a more clinically important outcome than simply measuring knowledge gain alone.
Only one of the six studies found significant improvement in knowledge gain, and the remaining studies found no significant difference or mixed effects. This supports previous findings of CAL efficacy in dental education.21 These results may be attributed to the many confounders in learning that exist and were not accounted or controlled for in the reported studies, such as the students’ level of motivation, previous knowledge of the subject, and learning style, as well as the teaching style of the educator.22 While all studies but one19 controlled for previous knowledge, none of the studies investigated these other variables. The one study19 that did not control for students’ previous knowledge reported no significant difference between CAL and traditional methods. Lack of control of this variable may have contributed to the different levels of improvement after receiving the intervention. In another study10 with a small sample completing the study, lack of information on sample size calculations puts in doubt the power of the study to detect a difference between the learning groups (total n=18). Therefore, results of both of these studies must be interpreted with caution. Of interest is the observation that two of the three studies that reported no significant differences received lower scores in quality assessment.
Clinical performance is an important outcome, but it was evaluated in only one study. In that study,20 CAL showed a significant effect on performing endodontics diagnosis but not treatment. The literature identifies many factors that influence clinical performance, including the student’s psychomotor skills, patient-related factors (e.g., patient comfort),23,24 treatment-related factors (e.g., difficulty of the clinical case being treated, treatment complications),23,25,26 and perhaps the type of CAL program used to learn the material (e.g., virtual-reality simulation versus a hypertext system). None of these factors were mentioned in the studies reviewed. That fact, combined with the limited evidence available, suggests that no conclusion can be drawn regarding the impact of CAL on clinical performance.
The costs of developing CAL were identified in two studies, but neither of them provided detailed information regarding operating and maintenance costs, nor did they perform comparative cost-effectiveness analyses. The cost and time to develop and structure CAL courses are reported as enormous.27,28 In an effort to control costs, Yip and Barnes28 suggested that institutions can “start small” and save costs by distributing the CAL programs to interested institutions nationally and internationally.
According to two studies, CAL proved to be more time efficient for students than traditional methods. Consequently, educators could use that time on other academic duties. A previous study reported that educators’ teaching time was cut by almost half after computer learning integration,29 but this outcome was not measured for educators in any of the studies reviewed. In addition, none of the studies correlated the time spent with the performance of students on tests.
Overall, students varied in their satisfaction with CAL. This finding is different from a previous systematic review21 of the effectiveness of CAL in dentistry that found that participants had positive attitudes towards CAL. The reaction to CAL is the first level of evaluation in Kirkpatrick’s four-level model. If participants did not have a positive experience, they would probably not accept the CAL method.
This systematic review had several limitations. Overall, the number of recent comprehensive studies published on the use of CAL in endodontics education is very limited and thus contributed to the small number of studies included in this systematic review. Also, despite the varied study population groups considered for inclusion in this review, only predoctoral students were evaluated in all the studies. This warrants the need for more studies in this area and the exploration of different CAL techniques as well as study populations. All studies lacked a reporting of power, important in evaluating the effect size, and did not include sample size calculations. Studies should calculate the sample size a priori and report power. Future studies should also consider evaluating other important outcomes, including diagnostic accuracy and knowledge retention, and evaluate different levels of the Kirkpatrick’s model, such as results, since none of the studies evaluated outcomes on this level. In some of the studies, other variables were not controlled for, such as pre-intervention knowledge, differences between instructors delivering the material when using traditional methods, and access to the intervention among the control group. CAL programs have the advantage of moving the learning process from pedagogical (teacher-directed) to andragogical (student-directed) through active learning.30 Based on this review, CAL is intended to support traditional learning methods, but not replace them.
Evidence from seven randomized controlled trials of adequate quality in endodontics education suggests that
CAL is as efficacious as traditional methods in improving knowledge.
There is some evidence to suggest that CAL is more time efficient than traditional learning methods.
In general, students have varied attitudes towards CAL.
Finally, two main issues regarding CAL in endodontics deserve consideration in the future. First, technology is ubiquitous and will continue to influence education. Second, whether CAL proves to be more efficacious than traditional learning or not, its advantage from other perspectives (e.g., learner satisfaction, time efficiency) have been demonstrated in previous studies in health care.31,32 Based on Friedman,33 the future suggests that we move from media-comparative research (comparing computer-based instruction [CBI] with non-CBI) to research that evaluates various designs of CAL, its pattern of use, and methods of assessment, as well as methods of CBI integration with current traditional learning. This may then allow the most effective use of this teaching technology.31
We thank Dr. Emad Al-Shwaimi for his comments on the manuscript and Dr. Prakesh S. Shah for his valuable advice.
Dr. Al-Jewair is an Orthodontic Resident, Department of Orthodontics, School of Dental Medicine, University at Buffalo; Dr. Qutub is a Dr.Clin.Dent. Candidate (Pediatric Dentistry), School of Dentistry, University of Adelaide, Australia; Dr. Malkhassian is Assistant Professor of Endodontics, Faculty of Dentistry, University of Toronto; and Dr. Dempster is Assistant Professor, Department of Biological and Diagnostic Sciences, and Kamienski Professor in Dental Education Research, Faculty of Dentistry, University of Toronto. Direct correspondence and requests for reprints to Dr. Thikriat S. Al-Jewair, Department of Orthodontics, School of Dental Medicine, University at Buffalo, 140 Squire Hall, Buffalo, NY 14214;.