Introduction

Quality of care is important, yet poorly defined for the emergency department (ED) setting, despite a vast amount of research into quality indicators [1,2,3]. Importantly, a wide variation of emergency medical system configurations across countries has limited the value of comparison of quality and performance measures in the past [4]. Furthermore, lack of quantitative data on EDs prevents definition of the problems and possibilities for data driven improvement of quality of care [5]. This is a concern because quality of ED care is continuously threatened by overcrowding with more complex older patients, shortage of medical and nursing staff and dynamic decision processes based on limited information on life-threatened patients [6, 7].

An international group of experts developed a template for uniform reporting of standardized measuring and describing of care provided in the ED [4]. The final measures and their definitions were 1) Structure, 2) Staffing and governance, 3) Population, 4) Process times, 5) Hospital and healthcare system, and (6) Outcomes. Although this template was applied on a national level [6, 8], it has not yet been used to compare ED systems across countries.

Currently, prehospital and ED care is organized differently across health-care systems [9,10,11]. For example, the number of general practitioners (GPs) and EDs per 100,000 inhabitants differs considerably across countries [11], and Australian EDs are run by emergency physicians, while in the Netherlands emergency physicians work together with other specialties. These differences may be justified to some extent because organization of ED care is partially determined or limited by political, cultural, financial and geographical differences. Nevertheless, organization of emergency medical services is often not evidence-based, potentially affecting outcomes. For example, reduction of the number of EDs in a health-care system can result in longer transport times to the ED, leading to higher initial disease severity of patients upon arrival in the ED, potentially affecting hospitalization and mortality. Conversely, a reduction in EDs will increase patient volumes into the remaining EDs, which in turn increases exposure of medical personnel. This may have a positive effect (practise makes perfect), or a negative effect (ED overcrowding).

Quality registries could be used to compare ED cohorts characteristics and relevant clinical outcomes using this template across countries. It would be an important first step to assess which data of this template are available in these registries. If such data are available and differences in relevant clinical outcomes exist, in-depth analyses of these differences could help to provide a more evidence-based approach to optimize emergency medical systems, strongly promoted in the World Health Assembly Resolution 60.22 [12].

The aim of this study was therefore to describe and compare ED cohorts, characteristics and their outcomes in three different health-care systems (Netherlands, Denmark and Australia), using the set of measures of a previously developed template for uniform reporting (structure, staffing and governance, population, process times and outcomes).

Methods

Study design

An international multicenter cohort study, using data from three quality registries from participating sites in the Netherlands, Denmark and Australia.

Study setting

For a detailed description of the study settings see supplementary file 1 and references [9, 13,14,15,16] for the Netherlands, [10, 17, 18] for Denmark and [19,20,21,22] for Australia.

The medical ethics review committee of Leiden Den Haag Delft (METC LDD)) declared that the research did not fall under the Medical Research Act, and waived the need for informed consent since this was an observational study (file no. G21.204). Low to Negligible risk ethics approval was granted for the study and use of data from the Western Sydney Local health District ethics committee.

Study population

All consecutive ED patients were included. See for inclusion dates supplementary file 2.

Data collection

Data were collected from three databases. Uniformity of variable definitions were discussed and checked by the researchers of the three participating countries.

Netherlands

The Netherlands Emergency department Evaluation Database (NEED) contained information of four hospitals between the period 1 January 2017 until 1 September 2021: one tertiary care center and three small to large urban hospitals.

Denmark

The Danish database contained information of all ED visits from patients above 18 years old in five different EDs (four individual hospitals) from 1 January 2016 until 19 March 2018.

Australia

The Australian database contained information of all ED visits in three hospitals and data of all adult ED patients (> 16 years) in one hospital from January 2017 to November 2019.

For a detailed description of data collection, see supplementary file 2 and 3 and reference [23]. Briefly, we collected demographic data, arrival by ambulance, involved specialties, triage categories and presenting complaints, vital signs, diagnostic tests. The top 18 presenting complaints were merged among the different triage systems (see supplementary file 4).

Sample size calculations can be found in supplemental file 5.

For exact variable definitions of data collected and shown in Tables 1, 2 and 3 see supplemental files 6 to 8.

Table 1 Patient characteristics per country
Full size table
Table 2 ED, hospital, healthcare and population characteristics according to Utstein framework per country
Full size table
Table 3 Relevant clinical outcomes per country
Full size table

Outcome measures

In-hospital mortality (including death on arrival or in the ED) and hospitalization (to any ward or unit including transfers to another hospital) were the primary outcomes. Secondary outcomes were ED and hospital length of stay, and number of ED revisits.

Statistical analyses

Data were presented as mean with standard deviation (SD) when normally distributed. Skewed data were presented as median with interquartile range (IQR). Categorical data were presented as number with percentages.

Differences among countries could only be calculated using aggregated data because data of individual patients were not allowed to cross borders. We used an online calculator to calculate differences of N (%) in Tables 1 and 3 (www.evanmiller.org/ab-testing/chi-squared.html). Outcome measures like in-hospital mortality were reported with 95% confidence intervals (95%-CI).

If 95%-confidence intervals did not overlap differences were considered to be significant. We adjusted the P value for multiple testing according to the method of Bonferroni. A P-value < 0.05 was considered to be statistically significant.

Data were analyzed using SPSS (SPSS, version 25.0, IBM, New York, USA) in the Netherlands and SAS (version 9.4) in Australia. In Denmark, STATA (version 16) was used to analyze the Danish data.

Handling of missing data

The percentage of missing data were provided in between square brackets in the tables and provided insight in the feasibility of fair comparison of countries. Missing data could be data which were not registered, i.e. vital signs are often not measured in patients with a twisted ankle. Impossible (based on expert opinion) or missing values were set to missing.

Results

Patient flow through the study

Figure 1 shows patient flow through the study. In total, 1,177,840 ED visits in three countries were included; 212,515 patients in the Netherlands, 408,673 patients in Denmark and 556,652 patients in Australia.

Fig. 1
figure 1

Patient flow through study

Full size image

Patient characteristics per country

Table 1 shows patient characteristics for each country. The median age varied widely among countries, being 39 (24–61) in Australia compared to 55 (25–73) in Denmark and 55 (29–72) in the Netherlands. The proportion of male patients was approximately 51%, comparable in the three countries. The proportion of patients triaged with the highest acuity level was lower in Australia (1.5%) compared to the Netherlands (3.1%) and different triage systems were used. In the Netherlands and Denmark mean oxygen saturation tended to be lower with a higher standard deviation compared to the Australian ED patients. However, the percentage of patients with a GCS lower than 15 was highest in Australia (3.1%) compared to 2.3% in Denmark and 0.9% in the Netherlands. The percentage of patients in whom no vital signs were measured was much lower in Australia (13.6%) compared to Denmark (61.2) and the Netherlands (34.2%). The percentage of patients who arrived by ambulance was similar in the Netherlands (28.4%) and Australia (28.1%).

The top 18 most common presenting complaints was similar across countries but their frequency differed with respect to several presenting complaints. For example, compared to Denmark and the Netherlands, the previously reported high-risk presenting complaint “feeling unwell”(23) occurred almost 50% less often in Australia. In the Netherlands, 8.4% had “dyspnea”, which was higher than in Denmark (5.9%) or Australia (4.5%). The proportions of “Chest pain” in the Netherlands (7.6%) and Australia (8.7%) were more than double of that of Denmark (3.6%). In Denmark, only 0.1% of the patients had the presenting complaint "collapse" compared to 2.4% in the Netherlands and 1.7% in Australia. Blood tests values were comparable across countries, but blood cultures were more often taken in Denmark (13.2%) compared to the Netherlands and Australia (both 7.7%). In the Netherlands radiological imaging was more often performed compared to Denmark (56.7% vs 30.1%).

ED characteristics per country

Table 2 shows the ED, hospital and health care characteristics per country. The number of GPs per 1000 inhabitants was higher in Australia (1.21) compared to the Netherlands (0.74), while the number of ambulance staffing per 1000 inhabitants was 0.38, similar in Australia and the Netherlands. The annual number of ED visits was higher in Australia (50.711 visits annually) compared to the Netherlands (26.738 visits annually). The percentage of geriatric patients was 11.6% in Australia, lower than in the Netherlands (18.9%). Finally, in Australia there was 24/7 staffing of the ED which was not the case in the Netherlands and Denmark.

Relevant clinical outcomes per country

ED length of stay for patients discharged home in the Netherlands was 2.1 h, much lower than the 3.3 h in Australia (Table 3).

In Table 3 it is also shown that the proportion of hospitalized ED patients was 33.8% in Australia and 30.6% in Denmark, lower than the 39.8% in the Netherlands. In Australian children aged 0–5 years 11.4 (11.3–11.5)% were hospitalized, compared to 29.3 (29.1–29.5) in the Netherlands.

The median hospital lengths of stay was 3 days in the Netherlands and Denmark, longer than the 1.8 days in Australia. In-hospital mortality was 4.0% in Australia, compared to 1.6% in Denmark and the Netherlands. In the geriatric population in-hospital mortality was more than four times higher in Australia (19.3%) compared to the Netherlands (4.5%). Finally, the number of 7-day ED revisits was 7.7% in Denmark, higher than the 4.8% in the Netherlands.

Discussion

Patient and ED characteristics and outcomes varied largely across countries, with a large variation of in-hospital mortality. Meaningful interpretation of these different outcomes is only possible if the measures proposed in the recently developed template for uniform reporting would be more consistently registered.

Studies investigating differences in patient, ED-characteristics and outcomes, using the recently developed template for uniform reporting according to the Utstein-style guidelines (4), are scarce. Two studies using the template on a national level showed that ED structure and staffing between EDs within one country differed considerably [6, 8]. We showed that ED systems across countries also varied largely. In addition, several studies comparing healthcare systems of different countries exist [11]. Baier et al. showed that large differences among countries exist in the organization of prehospital and ED care and kind of payment systems but the lack of use of a uniform template as used in the present study complicates direct comparisons of outcomes (11).

In the present study we attempted to compare countries in a more uniform way so that factors can be identified which could potentially be used to improve outcomes. However, not all measures of the template for uniform reporting were consistently registered in the quality registries. In addition, fair comparison of outcomes across countries requires appropriate case-mix adjustments. Because legislation does not allow data of individual patients to cross borders, multivariable regression cannot be used for such case-mix corrections. In the future case-mix adjustments maybe possible with federated learning techniques [24]. Development of appropriate models for case-mix adjustments with federated learning would probably require important variables like co-morbidity, frailty and obesity, which are currently lacking in the registries.

Patient characteristics at ED presentation

Nevertheless, some of the differences in patient characteristics at ED presentation are worthwhile to discuss because these characteristics provide information about the pre-hospital setting. Compared to Dutch and Danish patients, Australian ED patients were younger, were more often self-referrals (75% see supplemental file 1 compared to ~ 33% in the Netherlands, see www.stichting-need.nl), had a lower proportion of the highest acuity, and less often high-risk presenting complaints like “feeling unwell”(23). In addition, the proportion of patients discharged home from the ED was approximately 50% higher in Australia. The large proportion of self-referrals and apparent lower disease severity of the Australian population may be explained by the different financing and payment of pre-hospital and emergency care, while in the Netherlands and Denmark the GP may function more as a gatekeeper for the hospitals preventing many unnecessary ED visits. The observation that in the Netherlands and Denmark more patients present with the highest acuity level is more difficult to explain. On the one hand, it could be hypothesized that survivor bias plays a role in Australia where larger distances to hospitals may limit access to EDs explaining the lower proportion of patients with the highest acuity, but it is also possible that Danish and Dutch ED patients recognize their disease severity later and subsequently present to the ED sicker compared to Australian ED patients. It is also possible that the combination of keeping away unnecessary ED visits and not timely sending in sick patients by GPs might be an explanation for the higher acuity level in the Netherlands and Denmark. This could be a downside of the Dutch system in which GPs are gatekeepers of the hospitals. Only in-depths analyses could clarify these differences and identify opportunities for improvement in prehospital emergency care.

ED characteristics

In Australia, the annual census of ED visits and thus input of patients is two times higher compared to the European countries (Table 2), which could explain the longer through-put times as reflected in the longer ED length of stay of patients discharged home in Australia. Output of patients, i.e. exit blocks, are likely to be similar since the number of beds of 2.7 per 1000 inhabitants is slightly higher in Australia compared to Netherlands and Denmark (Table 2). If workload and the risk of ED overcrowding is higher in Australia compared to Denmark and the Netherlands, this may contribute to the differences in mortality [25], and may also contribute to the lower proportion of patients who are hospitalized compared to the Netherlands.

Relevant clinical outcomes

In-hospital mortality of ED patients in Australia is much higher than in the two European countries, especially in the geriatric population. Although it is possible that older patients in Australia are more frail or have more co-morbidity than in the Netherlands, it is likely that the selection of older patients visiting the ED is different in Australia. For example, in Australia it is less common for older people to die at home, instead people go to a hospital sooner, also if they have a DNR code. This information is currently lacking in the quality registries but is extremely important to take into account if meaningful conclusions about quality of care of the emergency medical services are to be taken.

In 2019 the Netherlands spent 10.17% of the Gross Domestic Product (GDP) on healthcare, which is slightly more compared to the 9.96% in Denmark and 9.42% in Australia [26], which may be partially explained by the markedly longer median length of hospital stay in the Netherlands compared to Australia. It is possible that the Dutch patients are admitted unnecessarily long compared to Australia. In the future, it could be examined whether this is indeed the case and whether healthcare costs can be reduced by shortening hospital length of stay. Interestingly, the annual census per ED is higher in Denmark compared to the Netherlands while in-hospital mortality is the same, suggesting that reduction of EDs with the argument that “practice makes perfect” is not necessarily applicable to the Dutch and Danish setting as has been suggested in a recent Danish study, despite the fact that this is recently suggested in policy statements in the Netherlands [27, 28].

The template was developed to function as a standard set of measures to make it possible to compare EDs or healthcare systems in a more uniform way. Although the framework was a helpful tool to compare EDs in different countries, we also showed that not all variables were available for each country limiting the possibility for comparison. Variables like number of nursing direct clinical care hours per 100 patient visits, would benefit from better definitions, because these variables are probably important for the quality of ED care. In addition, these variables are also important in order to be able to properly interpret the differences between countries. In the future, quality registries should register these variables in a reliable and consistent way.

Our study has several strengths, such as the large sample size in different countries and the use of the template for uniform reporting to compare countries. There are however also important limitations. First, this study has limitations inherent to the retrospective study design like the risk of information bias caused by human documentation errors and missing data. However, this risk is reduced by validating the data before using in the quality registers of the three contributing countries and the transfers of data are automatic. In addition, it was part of the feasibility aim of the study. Secondly, despite careful discussion and synchronization of variable definitions it cannot be ruled out that some variables have still a slightly different meaning in clinical practice. For example, in the Australian dataset 3 hospitals saw pediatric patients while 1 did not. In addition, it is possible that there are still some differences with respect to which patients are classified to certain categories of presenting complaints. Finally, it should be mentioned that some of the data are from 2016. It cannot be ruled out that in later years changes occurred in the participating EDs.

Conclusion

Large differences exist across countries with respect to patient and ED characteristics and their outcomes across countries. Future studies with more in-depth analyses of these differences in in-hospital mortality and hospitalization are needed to find modifiable aspects of health care which could be used to improve quality of ED care. These in-depth analyses would only lead to meaningful conclusions if the measures of the recently developed template for uniform reporting, and more detailed information for case-mix corrections are registered.