Introduction

Since the introduction of paper-based logbooks in 1983, anaesthetic trainees have had a structured system to document and assess their progress [1]. This system includes crucial details such as surgery types, patient risk factors, and level of supervision. It enables trainees to effectively track their cases and gain valuable learning experiences.

Logbooks allow trainees to reach their minimum competencies as set out by regulatory departments such as the Royal College of Anaesthetist [2]. From an assessor’s perspective, logbooks permit supervisors to assess whether the trainee receives the breadth of training opportunities required to become a competent and independent practitioner. This allows supervisors to tailor the learning objectives in future sessions to suit the needs of each trainee.

In addition, logbooks allow for the standardization of training programs nationwide, as demonstrated in England [3]. This is particularly important since postgraduate training experiences have been shown to vary based on geographical location [4, 5]. Therefore, the adequate completion of logbooks ensures equity in trainee competency and a reduction in the regional variation of patient care.

With the introduction of computerised records, trainees have begun to monitor their progress on their devices. In 2015, a survey of anaesthetic trainees in England found that 91% of respondents kept a logbook on their mobile devices [6]. Furthermore, recent technological developments have led to the proliferation of mobile and computer applications that can be used to log data. This allows for further standardisation of the data collected, making it easier for supervisors to process. Furthermore, data collected from these electronic logbooks can be analysed on a national basis to identify variations in case load and level of supervision by trainee grade and geographic location, thus allowing for the recognition of any disparity in patient care [3, 7]. An example of this was during the COVID-19 pandemic, when the number of logged cases drastically fell, with one logbook provider noting a 29.8% decline in cases recorded [8]. This data proves valuable as it allows supervisors regulatory bodies to adjust the training pathway to accommodate special circumstances. Such data can only be collected if trainees consistently engage with their logbooks, highlighting the need to assess the suitability of the current logbook and its completion rate.

Our postgraduate anaesthetic programme in Universiti Kebangsaan Malaysia (UKM) has made it compulsory for anaesthetic trainees to document their clinical cases encountered during their training using a logbook. Until 2019, all anaesthetic trainees used a hardcopy logbook, after which we introduced a softcopy alternative using Microsoft Excel. This study aims to assess the logbooks’ relevance in monitoring training and competency among UKM anaesthetic trainees.

Methods

Study design and population

This study was approved by the Research Ethics Committee of Faculty of Medicine, UKM (JEP-2023-230). This was a survey involving both quantitative and qualitative data. The quantitative part was designed to determine the relevance of each section in the logbook and to evaluate the user’s experience of using the logbook. The qualitative section explored trainees’ opinions on ways to improve the logbook. The study was conducted among all postgraduate anaesthetic trainees of UKM who have consented to partake in this study from academic year 1 to year 4. Year 1 and year 2 anaesthetic trainees were placed in other health care centres around Malaysia while year 3 and year 4 trainees were placed in Hospital Canselor Tuanku Muhriz (HCTM), UKM. Consent to participate was obtained from all of the participants in this study.

Study location and duration

This survey was conducted at HCTM, UKM, Kuala Lumpur and other health care centres around Malaysia where UKM anaesthetic trainees are placed in. The study took place from March 2023 to August 2023.

Study protocol

All anaesthetic trainees from year 1 to 4 were recruited for this study, while those who deferred their studies were excluded. The survey data was gathered through an online Google Form, with survey links distributed via email to all anaesthetic trainees. This approach facilitated convenient data collection, accommodating trainees in various hospitals as part of their training attachments. To ensure comprehensive participation, a follow-up reminder email was sent two weeks after the initial contact, with a final reminder sent four weeks later.

The survey questionnaire comprised five sections, covering demographic data, strengths and weaknesses of logbook content, evaluation of logbook usage experience, perception of the feedback received regarding logbook performance, and potential areas for improvement. The questionnaires used was adapted from two different sources by Ahmad Abdul Azzzem Abdullah and Viseskul et al. which had limited usage and required further validation, as they we merely survey questions [9, 10]. In our extensive review of the literature, we found no validated tools that precisely met our study’s objectives and context. Despite their limited use, the questionnaires by Abdullah and Viseskul et al. are robust and well-designed. By merging these two sets, we have created a comprehensive tool that is better suited to address the unique needs and characteristics of our local trainees. We opted for the 5-point Likert scale of strongly disagree to strongly agree, which differed from the previous studies [9, 10]. Given this adaptation, this questionnaire underwent content validation by six content experts from medical education and anaesthesiology looking at its accuracy, comprehensiveness and appropriateness. The scale level content validity index (S-CVI) and the item level content validity index (I-CVI) was 0.9 and 0.98 respectively. The Cronbach alpha was 0.93, which was calculated as part of questionnaire validation. Additionally, at the end of the survey, we incorporated an open-ended question aimed to elicit respondents’ perspectives on potential improvements that can be made to the logbook.

Sample size calculation

Sample size was calculated using the Krejcie & Morgan formula for a finite population where the number of trainees was 127. Based on the Viseskul et al. study, 91.2% of participants showed overall satisfaction with their logbook (N = 127, P = 91.2%, d = 5%, X2 = 3.841 for 95% confidence level) [10]. Therefore, 80 subjects were the minimum sample size to be recruited with 80% power of study, 95% confidence level, and anticipation of 20% dropout rate.

Statistical analysis

All calculations and statistical analyses were performed by using Statistical Package for the Social Sciences (SPSS) version 29. Descriptive statistics were used for demographic data and Likert scale questions, while free-text answers were analysed qualitatively using abductive reasoning. Descriptive statistics included measurements of frequencies (n) and percentages (%) were presented using tables. Stacked bar graphs were also used to present the Likert scale results for each question.

Kruskal-Wallis test for independent samples assessed the relationship between responses to Likert scale questions and demographic data. In this analysis, responses to each question served as the dependent variable, while demographic categories were used as the independent, or grouping variable. For cases where the relationship was statistically significant, post-hoc analyses with the Dunn test was carried out to identify the distinct groups that differed. Additionally, to address the issue of multiple comparisons and reduce the Type I error rate, p-values were adjusted using the Bonferroni correction method. A p-value of less than 0.05 was considered to be statistically significant. The data obtained from this study can only represent the population of UKM anaesthetic trainees in 2023.

Results

Demographic criteria

A total of 107 UKM anaesthetic trainees responded to our questionnaire and their demographic criteria presented in Table 1.

Table 1 Demographic criteria of respondents among UKM anaesthetic trainees
Full size table

Strength and weakness of logbook content

The second part of the questionnaires explored the opinion of the anaesthetic trainees’ opinions regarding the relevance of each section in the logbook. As shown in Table 2, the top three sections perceived with the highest relevance are the supervisor’s verification Sect. (84.2%), followed by the anaesthetic case record Sect. (83.2%) and the intensive care case record Sect. (81.3%). In contrast, the most irrelevant section perceived by the respondents is the on-call roster Sect. (29.9%).

Table 2 Strengths and weaknesses of logbook content
Full size table

Evaluation of logbook usage experience

The third part of the survey explored the respondents’ experience of using the logbook. 53.3% agreed that the logbook is useful for learning during the course. However, less than half of the respondents perceived that the logbook is easy (43.9%) and convenient (43.9%) to use, as shown in Fig. 1.

Fig. 1
figure 1

Perception on logbook usage experience

Full size image

Opinion regarding the feedback received on logbook performance

The subsequent section delved into participants’ perceptions of the feedback they received on their logbook performance, as detailed in Table 3. Out of 107 respondents, only 59 received feedback at the end of the semester. Among these 59 anaesthetic trainees, 54.3% agreed that the feedback had enhanced their learning capabilities, while 64.4% acknowledged that it had underscored deficiencies in the quantity or type of essential anaesthetic procedures.

Table 3 Perception of the feedback received on logbook performance
Full size table

The Kruskal-Wallis test for independent samples revealed statistically significant variations in Likert scale scores across different years of postgraduate training for five survey questions, as detailed in Table 4. The post-hoc Dunn-Bonferroni test was employed to identify specific years of postgraduate training that exhibit differences between academic years. Higher scores represent greater agreement, while lower scores denote greater disagreement. Across all five questions, year 2 anaesthetic trainees reported significantly lower scores than year 1, indicating a lower agreement with the statements among second-year trainees. Similarly, year 2 anaesthetic trainees expressed lower agreement that the logbook is easy to fill compared to those in the year 3. Moreover, Likert scores for year 2 were significantly lower than those of years 3 and 4 when assessing the logbook’s ease of use. Additionally, there was a notably lower level of agreement among year 2 anaesthetic trainees, regarding the perception that the feedback has highlighted inadequacy of the amount or type of core anaesthetic procedures required, compared to year 3 anaesthetic trainees.

Table 4 Statistical significance of likert scale score distribution between different years of postgraduate training apost-hoc significance values have been adjusted by the Bonferroni correction for multiple tests
Full size table

Potential improvements of anaesthetic logbook

The last part of the questionnaire was an open-ended question asking on any opinion on how the logbook can be improved for anaesthetic training. The most (n = 46) received answers were suggestions to simplify the logbook, easier to be fill in immediately instead of retrospectively, and easily accessible. A few suggested changing the logbook form into a mobile application, Google Forms or an online system. Below are some of the trainees’ comments:

Make a more simplified version of the logbook that can be filled up easily daily while at work.

Online system will be good, we can update on the go using google documents. I’ve been using online format for my own logbook and find it quite easy and conducive.

I think using an online software to register for all anaesthetic and ICU patients entree will be more convenient and easy to refer again later.

Can make it into an application for ease of use.

Hopefully someday we would have an anaesthesia logbook training application on mobile devices that can make documentations and analysis easy.

Make a system to automatically and immediately transfer details from OT list/ICU census every day.

Ability to key in details in offline method using dedicated university app, with more simplified and direct interface, and will be updated once allowed to synchronize with database, and export for offline usage.

No need to enter manual diagnosis and operations. It is nicer if can just checked boxes and logbook in application form is more relevant.

Digitalisation and extract from hospital data base.

Other than that, several trainees suggested to omit the on-call section from the logbook as supposedly all trainees should have the same number of on-calls. This was evident in some of the respondents’ comments such as:

There is irrelevance for on-call section for registrar post in ministry of health hospitals.

Omission of total calculation of number of on-calls per month.

Distribute the on-call evenly among the colleagues.

Although minimal, there are also few respondents who expressed their disagreements on the usefulness of logbook during the training. They prefer other method to replace the logbook-based assessments. The following quotes demonstrate this opinion:

We hope to enjoy the rotation instead of filling up logbook. I understand that logbook is important for recording anaesthetic cases, but one year one logbook is a burden.

A section for some directly observed procedures may be more beneficial in comparison to a long repetitive census of procedures.

Put less emphasis on filling up logbook but more towards practical and exam orientated training.

For my opinion, logbook is irrelevant instead evaluation as per semester by supervisor is more important. Anyone can copy any patient to the logbook, it’s not the number that we want to see but how one is performed.

Experience is more important than logbook.

Discussions

The existing logbook utilized by our anaesthetic trainees is a softcopy version, requiring entries using Microsoft Excel, making it not fully digitalized. Our study revealed that less than half of the respondents perceived the anaesthetic logbook as easy (43.9%) and convenient (43.9%) to use. This is in consistent with the free-text answers of which a significant number of them put forward ideas of changing the logbook into an online or application form for easier use and access. The transition from a traditional logbook to fully digital logbook will allow immediate recording from any location. This instant accessibility is crucial for the dynamic environment of medical training [7, 11,12,13,14]. Additionally, digital logbooks provide a standardized format for recording clinical experiences, ensuring consistency and ease of documentation [11, 12]. Integrating digital logbook with the hospital’s electronic health records, enabling automatic transfer of census data offers significant advantages [15]. Automatic data transfer reduces the risk of manual entry errors, ensuring more accurate and reliable records. Furthermore, real-time access to updated information improves decision-making, while streamlined data management facilitates easier storage, retrieval, and analysis [15]. However, integrating digital logbooks with existing electronic health records and other educational platforms can pose technical challenges. Implementing an entirely new system necessitates comprehensive training for both students and faculty. Ensuring user-friendliness and providing adequate technical support are critical for successful implementation of digital logbooks.

When respondents were questioned about the relevance of each section in the logbook, over 60% agreed that almost all sections to be pertinent. However, the on-call section received a lower agreement of relevance, with only 37.4%. According to the free-text answers, this discrepancy is attributed to the uniform scheduling of on-calls for all trainees in their respective workplaces. Thus, recording their number of on-calls is unnecessary. Feedback of this nature is crucial for continuous refinement of the curriculum for both lecturers and trainees [16, 17]. According to The National Medical Commission (NMC), the training structure for anaesthesiology postgraduate program requires specified objectives to meet the goals of producing competent specialists [18]. Therefore, each component of this logbook must underline its purpose and specific goals. While it is true that the on-call section is essential for ensuring adequate exposure, it may be deemed unnecessary if the number of on-calls for each trainee is fixed according to the hospital schedule. Faculty members may need to reassess the importance of this section.

When statistical analysis was conducted to examine the relationship between responses to Likert scale questions and the year of postgraduate training, this study revealed that year 2 students provided significantly lower scores compared to year 1, year 3 and year 4 students. Year 1 students tended to give higher scores compared to year 2 students, suggesting a more positive perception of logbooks at the outset of their training. This initial positivity in year 1 may be attributed to several of factors: a fresh perspective on educational tools, eagerness to engage with new learning methods, and potentially less exposure to the challenges or limitations of logbook systems. In contrast, by year 2, students may have encountered more practical difficulties or perceived inefficiencies in traditional logbook methods. These challenges could include concerns about time-consuming data entry, insufficient or delayed feedback, and a growing preference for digital tools that offer real-time data access and more interactive learning experiences.

Younger medical trainees, who are often more accustomed to digital technologies, may consider traditional logbook methods as outdated, inefficient, and burdensome when compared to fully digitalized form [7, 11,12,13,14]. Moreover, the culture of feedback in medical training has evolved, and current trainees may expect more constructive, timely, and personalized feedback, which they may perceive as lacking in traditional logbook systems that provide feedback at the end of the semester [16, 17, 19, 20]. Additionally, younger trainees are typically in the early stages of adapting to the rigorous demands of medical training. Hence, they might question the relevance or usability of logbooks, favouring methods they perceive as more directly beneficial to their learning [9, 21]. Compared to more senior trainees, having more experience allows them to have more accurate judgements regarding the nuances of the logbook system. Nevertheless, it is essential to highlight that a previous study showed that the enhanced accuracy in judgement might, nonetheless, lead to a potentially more negative perception when compared to their junior counterparts [9].

While most of our anaesthetic trainees acknowledged the utility of the logbook for learning during the course, some contended that alternative methods, such as directly observed procedures and assessments, offer greater benefits for their training. This perspective may stem from the perceived burden of daily logbook entries, a challenge that could be alleviated by providing a user-friendly digitalized logbook. The logbook itself is an important component to track the competency of trainees especially before the conferral of any qualifications. Beyond mere recordings, there exist intricate processes of inspections, monitoring, and analysis aimed at ensuring and maintaining adequate exposure and skills [1,2,3,4,5, 16]. Logbook itself is a key method within the competency-based approach, offering healthcare professionals opportunities to enhance competency training and engage in self-reflection. It serves as a tool that bridges the gap between theoretical knowledge and practical application, effectively preparing healthcare professionals to tackle the intricate challenges of modern healthcare delivery. Logbook can also contribute to standardising and upholding the quality of the curriculum. Recognising this, regulatory bodies such as the NMC or Royal College of Anaesthetists could develop a standardised logbook for use by all anaesthetic trainees globally.

Given this perspective, the suggestion to include directly observed procedural skills (DOPS) in the assessment of our anaesthetic trainees should be considered. This approach provides an opportunity to directly assess trainees in clinical settings through a structured checklist [22]. Various studies emphasize the efficiency of DOPS as an assessment tool, citing its validity and reliability, coupled with its positive impact on learning among postgraduate students [23, 24]. The benefits of this evaluation method encompass immediate feedback to trainees, which stands in contrast to feedback at the end of the semester on logbook performance, in addition to its inherently structured nature.

Despite the expectation of feedback sessions with supervisors on trainees’ logbook performance, only 55.1% of respondents in this study received the feedback at the end of the semester. Feedback plays an important component in teaching and learning process shedding light on students’ shortcomings to enhance their capabilities [9, 16, 17, 19, 20]. More than half of those who received feedback in this study agreed. Engaging students to reflect upon their performance during a debrief followed by actionable guidance provided by the teacher has shown to help improve students’ clinical performance [25]. Recognising this, there is a need for proper implementation of feedback mechanisms in our anaesthetic training logbook.

Challenges and limitations

Researching improving the UKM anaesthetic logbook involves several limitations and challenges. Although free text comments are applicable, the best experimental design is to conduct a focus group discussion, securing participation from both students and lecturers is difficult due to their busy schedules, requiring flexible coordination and incentives [26]. Diverse perspectives can lead to conflicting input, necessitating skilled facilitation to ensure all voices are heard and synthesized into cohesive recommendations.

Next, technological barriers may hinder the transition to a fully digitalized logbook, highlighting the need for comprehensive training and support. The implementation costs of developing and maintaining a digital logbook system also present a significant challenge, necessitating thorough funding proposals and pilot programs to justify the investment [16, 27]. Resistance to change from both students and faculty requires clear communication of benefits and addressing concerns through effective change management strategies.

Designing effective, timely, and actionable feedback mechanisms is essential for integrating feedback into the logbook system to improve training outcomes. Achieving standardization while accommodating different training programs involves balancing flexibility and customization [28, 29]. Evaluating and updating the logbook content requires thorough assessments and consensus on necessary changes.

Finally, conducting a longitudinal study to assess the impact of logbook improvements demands significant time and resources. Sustained engagement strategies are needed to maintain participant involvement and address external variables. Addressing these challenges through strategic approaches can yield valuable insights for developing a robust, efficient, and standardized logbook system that enhances postgraduate training programs.

The way forward, replicating this experimental study on improving the anaesthetic logbook system at institutions outside UKM, involves several strategic steps to adapt the methodology to different educational contexts. First, securing institutional commitment is crucial, emphasizing the potential benefits for training outcomes. Engaging stakeholders, including students, faculty, and administrative staff, ensures their participation and input. Customized focus groups with diverse participants should be organized, with skilled facilitators managing discussions to gather balanced input. Assessing the current technological infrastructure and providing necessary training can facilitate the transition to a digital logbook [16, 27]. Implementing pilot programs allows us to thoroughly test and fine-tune the digital logbook system, identifying potential issues and making necessary adjustments before rolling it out on a larger scale. This approach ensures that the system is both effective and user-friendly when it reaches full-scale implementation. Incorporating effective feedback mechanisms into the logbook ensures timely and actionable feedback, fostering continuous improvement [28, 29]. Robust data privacy and security measures must be implemented to protect sensitive information and ensure compliance with local regulations. Developing a standardized logbook framework that allows customization for specific training programs ensures consistency while addressing unique educational needs. Our findings also showed that regular reviews and updates of the logbook content, based on user feedback and evolving standards, are essential for maintaining relevance. Other institutions can effectively use this data to further enhance their postgraduate training programs and ultimately improve medical education and patient care.

Conclusion

The logbook system guides trainees to reach their minimum competencies required and allows standardisation of training programs. In this study, the UKM anaesthetic logbook has been extensively explored, revealing significant insights to aid in updating and further improving the current logbook. The results presented highlights the importance of transitioning to a fully digitalize logbook to ensure easy accessibility and convenience. Discussion of the logbook content at the beginning of the clinical training are essential to assist understandings of the logbook’s objectives. Moreover, based on the survey results, the faculty members might need to re-evaluate certain contents and add any other teaching-learning tools in the logbook. Most importantly, properly implementing feedback mechanisms is needed to optimize students’ learning from the logbook.

This survey will provide the first overview of our current logbook and can serve as an initial proposal improvement to a more efficient, structured and standardised tool to assess our postgraduate training programme. This serves as a stepping stone towards further advancement in medical education leading to better training outcomes and more competent medical professionals. Future research of a longitudinal study could be done to assess the impact of logbook improvements on training outcomes.