Abstract
Background
Ventral hernias (VHs) often recur after surgical repair and subsequent attempts at repair are especially challenging. Rigorous research to reduce recurrence is required but such studies must be well-designed and report representative and comprehensive outcomes.
Objective
We aimed to assesses methodological quality of non-randomised interventional studies of VH repair by systematic review.
Methods
We searched the indexed literature for non-randomised studies of interventions for VH repair, January 1995 to December 2017 inclusive. Each prospective study was coupled with a corresponding retrospective study using pre-specified criteria to provide matched, comparable groups. We applied a bespoke methodological tool for hernia trials by combining relevant items from existing published tools. Study introduction and rationale, design, participant inclusion criteria, reported outcomes, and statistical methods were assessed.
Results
Fifty studies (17,608 patients) were identified: 25 prospective and 25 retrospective. Overall, prospective studies scored marginally higher than retrospective studies for methodological quality, median score 17 (IQR: 14–18) versus 15 (IQR 12–18), respectively. For the sub-categories investigated, prospective studies achieved higher median scores for their, ‘introduction’, ‘study design’ and ‘participants’. Surprisingly, no study stated that a protocol had been written in advance. Only 18 (36%) studies defined a primary outcome, and only 2 studies (4%) described a power calculation. No study referenced a standardised definition for VH recurrence and detection methods for recurrence varied widely. Methodological quality did not improve with publication year or increasing journal impact factor.
Conclusion
Currently, non-randomised interventional studies of VH repair are methodologically poor. Clear outcome definitions and a standardised minimum dataset are needed.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
In the UK, 44,000 ventral hernia (VH) repairs were performed in 2010, increasing to nearly 50,000 in 2015, a 13% rise over 5 years [1]. With an ageing [2] and increasingly obese [3] population, the risk of incisional hernia post midline laparotomy has increased, from 8% in 1980 to 16% in 2012 [4]. Recurrence after a previous hernia repair is also high, with minimal improvement over the last 30 years [5]. VHs that repeatedly recur, have a wide ventral defect or are contaminated are known as complex VHs, and successful repair is extremely challenging [6].
This surge in prevalence and complexity of VH disease has attracted attention from academic surgeons and given rise to specialised university hernia centres [7]. As VHs are predominantly iatrogenic, it behoves surgeons to investigate both prevention and cure. This demands high quality research to generate robust and meaningful data. We have recently investigated the methodological quality of randomised controlled trials (RCTs) of VH repair [8] and found that studies frequently employed poor methods, risking bias. We discovered that studies collected highly variable data relating to the pre-, intra-, and post-operative variables and reported multiple poorly defined outcomes. In particular, there was no standardised definition for hernia recurrence, length of follow-up, or methods to diagnose recurrence. This current variation in reported perioperative variables and outcomes frustrates comparison of outcomes across different trials. These challenges would be greatly diminished if investigators adhered to a common set of reported variables and outcomes. Consequently, there is an urgent need to establish a standardised minimum dataset for trials of VH repair. Adopting such a dataset would facilitate data pooling and allow researchers to better explore the impact of patient demographics, hernia characteristics, and intra-operative variables on both operative and patient outcomes.
The fact that some surgical studies lack methodological rigour has been identified previously and a recent systematic review found that 62% of surgical journals do not require authors to adhere to recognised reporting guidelines [9]. Reporting tools have been designed specifically to enhance reporting of surgical interventions [10]. For this methodological review of non-randomised interventional studies in VH repair we designed our own methodological assessment tool for VH studies using a combination of reporting guideline tools already published (Downs and Black [11], ROBINS-I [12], Newcastle–Ottawa [12], TIDieR [10] and STROBE [13]) and our own expert knowledge of the VH literature.
The aim of this systematic review was to evaluate the methodological quality of non-randomised interventional studies of adults undergoing VH repair. We hypothesize that there is a lack of rigorous research in VH repair studies, as demonstrated in the aforementioned review of VH RCTs [8]. We further aim to establish evidence from non-randomised studies, that clear outcome definitions along with a standardised minimum dataset are required in this field of surgical science.
Methods
Registration and reporting
This systematic review is reported in line with the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement [14]. Ethical permission is not required by our centre for systematic reviews of available primary literature. A protocol was developed and registered with PROSPERO, the international prospective register of systematic reviews (CRD42016043071).
Eligibility criteria
Study design
We included non-randomised interventional studies of VH repair. We anticipated finding fewer prospective than retrospective studies. To compare their methodological quality, we included all eligible prospective studies identified, matching each with a single retrospective study.
Participants
We included studies of adults. We excluded paediatric studies (defined as 18 years or less) since these are no representative of ‘typical’ VH patients. As our review was methodological, we included all hernia populations and included studies than restricted participants according to specific diseases, conditions, or metabolic disorders (e.g. a study of participants with BMI > 30).
Target condition
We defined VH as any anterior abdominal wall defect associated with abnormal protrusion of intra-abdominal viscera [15]. We, therefore, included a range from simple primary umbilical/epigastric to large complex hernias. Studies combining multiple types of hernia were eligible, as we were interested in how hernias were graded.
Interventions
All interventions addressing VH repair were eligible. So, we included all types of comparative study, including those comparing mesh, plane of mesh insertion, surgical technique, with/without component separation, with/without panniculectomy, etc. Studies comparing the same intervention with minimal alteration were also eligible (e.g. “double-crown” versus “single row” tacks for laparoscopic repair).
Comparators
All interventional comparators were eligible. Studies that compared an intervention to conservative management (i.e., non-operative management of VH) were excluded.
Outcomes
Any study outcome was eligible.
Timing
We stipulated no minimum follow-up.
Setting
All settings were eligible.
Language
We restricted our search to the English language.
Information sources
We searched the PubMed database (US National Library of Medicine, National Institutes of Health, Bethesda MD, 20894, USA) from 1st January 2005 to 1st January 2018. Our prior experience of systematic review of clinical interventions suggests that this is the most comprehensive database and little additional benefit is gained from searching other databases.
Search string
Our search string identified and combined the two following criteria:
-
1.
To identify studies of VH disease including complex disease we used the MESH terms “hernia”, “abdominal hernia”, “umbilical hernia” and “ventral hernia”. These were combined with keywords: “abdominal wall reconstruction”; “herniorrhaphy”; “ventral defect” and “entero-cutaneous fistula”.
-
2.
To identify studies of surgical techniques used for VH repair we used the MESH terms: “general surgery”; “reconstructive surgical procedures” and “surgical mesh”. This was combined with keywords: “pneumoperitoneum”, “botox”, “botulinium”, “two-stage”, “two step”, “staged repair”, “component separation”, “transversus abdominis”, “retro-rectus”, “bridging”, “bridge repair”, “silo”, “open” and “laparoscopic”.
Our complete search string is shown in Online Supplementary Material 1.
Study records
Data management
Identified citations were entered into a spreadsheet (Microsoft Excel for Mac 2011 v. 14.5.9, Microsoft Corporation, Washington), and uploaded subsequently into a reference manager able to access online original articles directly (Mendeley Desktop v. 1.17, London, UK).
Citation management and screening
Citations were divided up into two equal groups. The first-half were screened by (SGP) and the second-half by (CPJW), both surgical fellows. They discarded articles that were “clearly unsuitable” (e.g. subject not VH), retaining any regarded as “uncertain” or “definitely possible”. These two latter groups were then combined and all assessed independently by SGP, CPJW, and RWB to identify all eligible studies. These were divided into methodological groups as follows: randomised controlled trials, non-randomised prospective interventional studies, non-randomised retrospective interventional studies. Any article where uncertainty persisted was discussed face-to-face with senior members of the research team (SH and SM). An exclusion log was kept at all stages.
The randomised controlled trials were excluded from the present review and reported elsewhere [8]. The following data were extracted from remaining studies; journal, impact factor, and publication year. Each prospective study was matched to a retrospective study. We attempted to match each prospective study to a retrospective study published in the same journal and year. If no studies met this criterion, we matched to retrospective studies published in the same journal but not in the same year. If no relevant articles were published in the same journal, we matched the prospective study to a retrospective study published in a journal with the closest impact factor. This procedure created a group of matched prospective and retrospective studies. A log of the matching process was kept. The flow of article selection is shown in the PRISMA diagram (Fig. 1).
Data extraction
SGP and ME extracted data independently from selected studies. To ensure consistency, data were cross-checked subsequently face-to-face and disagreement resolved by a third author, CPJW, and by senior authors, SH or SM, if discrepancy persisted. Data were entered into an Excel datasheet and categorised into broad groups as follows: introduction, study design, participants, reported outcomes, and statistical analysis.
Data items
To assess methodological quality, we designed a methodological assessment tool relevant to our review by combining the most important data points from the following reporting and risk of bias guidelines tools: TIDieR [10], Downs and Black [11], ROBINS-I [12], STROBE [13], Newcastle–Ottawa [16]. Our tool is described in Online Supplementary Material 2. To analyse the introduction, we attempted to identify a rationale, primary aim or objective, and a pre-specified hypothesis with references to existing literature. To analyse design, we identified whether data were collected prospectively and according to a protocol. We also analysed whether studies described the equipment used and the proposed intervention adequately, using pre-specified criteria (Appendix 1 and 2, Online Supplementary Resource 2). We identified whether a primary outcome was described and whether a sample size calculation had been performed.
Regarding participants, we identified how patients were selected. We identified whether participants’ selection criteria or process was described adequately, and whether participants in intervention and comparator groups were drawn from the same population. To assess selection bias and to differentiate between patients meeting inclusion criteria versus number of participants included, we identified whether the study reported eligibility. We collected data on hernia morphology, assessing previous repairs were reported, maximal hernia width, defect area, whether primary or incisional hernias were reported, and whether a hernia grading scale was used. To assess participant characteristics, we identified whether a table of basic demographics was reported according to pre-specified criteria (Appendix 3, Online Supplementary Resource 2). To assess participant recruitment, we recorded whether recruitment start date, finish date, and end of follow-up date were reported. We identified whether the number of participants deviating from the intended intervention was reported.
Regarding reported outcomes, we assessed whether the assessor and/or participant were blinded to the intervention. Remaining information collected under this heading related to primary and secondary outcomes (see sections below).
For statistical analysis, we identified whether median length of follow-up and the number of participants with missing data were reported. We identified whether an adjusted analysis was performed and whether any adjustment factors were reported. We identified whether prediction estimates were reported for standard clinical variables. We also assessed whether confidence intervals were stated for all reported estimates. We identified whether an intention-to-treat or complete case analysis had been performed since this is most realistic in the clinical setting.
Outcomes and prioritization
Our primary outcome of interest was hernia recurrence, so we extracted post-operative recurrence rates. We also extracted the timing of recurrence, definitions for VH recurrence, and the test method(s) used for diagnosis (for example, clinical examination, CT scan, and US scan). Our secondary outcomes were surgical site infection and surgical site occurrence, and we extracted definitions used to define them in the component studies. We also assessed whether a patient reported outcome measure was reported and, if so, its identity. Finally, manuscripts were reviewed to see whether a visual analogue scale (VAS) was used to assess post-operative pain.
Risk of bias in individual studies
Existing reference tools were analysed [10,11,12,13, 16] and our assessment tool designed to identify the following categories of potential bias:
-
1.
To assess selection bias we identified whether a study reported the number of eligible versus included participants.
-
2.
To assess bias from intervention classification we included two questions from the TIDieR assessment tool [10]: (1) was a detailed description of equipment used reported (according to Appendix 1, Online Supplementary Resource 2)? And, (2) was a detailed description of the intervention reported (according to Appendix 2, Online Supplementary Resource 2)?
-
3.
To assess bias regarding outcome measurement, we identified whether participants and/or assessor were blinded to the intervention.
-
4.
To assess missing data bias, we identified if analysis was restricted to patients with full data.
Studies were assumed to be at low risk of bias if they adhered to all these criteria. ‘Unclear’ criteria were classified as moderate risk. ‘High’ risk of bias was determined by clear non-adherence to any criteria.
Data synthesis
We used descriptive tables of frequencies for study items for prospective and retrospective studies. Box and whisker diagrams were used to present total methodological scores and to compare prospective and retrospective studies, enabling us to assess overall methodological quality. Scatter plots showed whether methodological quality was related to publication year and/or impact factor.
Results
Search results
Our initial search retrieved 11,316 results (Fig. 1). After applying filters (studies published 1st January 2005 to 1st January 2018; human; age > 18; English language), we excluded 5370 studies, leaving 5946. After title screening, 640 studies were categorised ‘definitely possible’ or ‘uncertain’, falling to 152 after abstract screening. After full text assessment, there were 119 non-randomised interventional studies; 25 prospective, 94 retrospective. Thus, after matching the prospective studies as described previously, the final review comprised 50 studies in total.
Study demographics
Study demographics are shown in Table 1. The 50 studies reported 17,608 patients overall, 2800 (16%) prospective studies and 14,808 (84%) retrospective. Twenty-one studies (42% of total) were from the United States; 17 retrospective and 4 [17,18,19,20] prospective. Just five (10%) studies were multi-centre [21,22,23,24,25]. There were five categories of study with the same comparison groups: Nineteen laparoscopic versus open repair, five mesh versus suture repair [26,27,28,29,30], two primary fascial closure versus bridged repair [31, 32], two heavyweight versus lightweight mesh [33, 34], and two endoscopic component separation versus open component separation [35, 36]. Twenty-one (42%) studies (8 prospective, 13 retrospective) reported compliance with national or regional ethical standards. Three (6%) prospective studies [28, 37, 38] reported approval from an ethics committee, 3 more (6%) [18,19,20] referenced approval from the institutional review board, 1 (2%) study [39] reported ‘compliance with ethical standards’, and 1 (2%) study [40] reported compliance with ‘National Patient Rights Regulations’. Twelve (24%) of the retrospective studies reported approval from the institutional review board and 1 (2%) [29] reported approval from the hospital research ethics committee. Hernia type was specified by 32 (64%) studies; 18 prospective, 14 retrospective. Thirteen studies analysed both primary ventral and incisional hernia, eleven analysed incisional hernia, 3 analysed primary incisional hernia only [20, 38, 41], 3 analysed primary VH [27, 42, 43] and 2 analysed primary umbilical hernia only [28, 44].
Risk of bias assessment
All studies were rated as at high risk of bias. Figure 2 shows that this was mostly due to unblinding of both participants and assessors; only three (6%) studies [19, 47, 60], all prospective, achieved blinding for both these criteria. Although we aimed to assess selection bias, only six studies reported patient eligibility; four prospective [38,39,40, 60], two retrospective [29, 48].
Methodology scores
Online supplementary resource 3 shows tabulated results from data extracted.
As our data extraction sheet had 46 items, the maximum possible methodology score for any single study was 46. Total and sub-category median methodology scores with their interquartile ranges (IQRs) are depicted using Box plots in Fig. 3. The overall median score was 16 (IQR: 14 to 18), with a range of 11 to 31. Prospective and retrospective studies had median and IQRs of 17 (IQR: 14–18) and 15 (IQR: 12–18), respectively, with prospective studies having marginally better average methodological quality. For the sub-groups ‘introduction’, ‘study design’ and ‘participants’ prospective studies achieved higher median scores relative to the matched retrospective studies with median scores of 2 vs 1, 2 vs 1, 7 vs 6, respectively. For the subgroup ‘reported outcomes’ prospective and retrospective studies had equal median scores, 4 vs 4. In the ‘statistics’ subgroup the retrospective and prospective median scores were 2 vs 1 (Fig. 3). Scatter plots of methodological quality against publication year and impact factor (Fig. 4) showed no clear relationship for either prospective or retrospective studies. One study, Kurmann et al. [60], scored 31 and was 8 points higher than the next best methodological score.
Introduction
All 50 studies (100%) provided a scientific rationale for their purpose. Twenty-nine studies (58%) described a primary aim or objective, with improved reporting for prospective (18 studies, 72%) versus retrospective (11 studies, 44%) studies. Only 3 studies [17, 32, 48] provided a hypothesis, and none of these referenced their hypothesis to the literature.
Study design
No study (0%) stated that a study protocol had been published or written. Studies were generally poor at accurately describing the equipment used for hernia repair but were informative about the interventions performed. Nineteen (38%) and 36 (72%) studies reported these criteria, respectively. Only 18 (36%) studies defined a primary outcome, with similar proportions for prospective and retrospective studies; 8 (32%) vs 10 (40%). Only 2 (4%) studies performed a power calculation [38, 47].
Participants
Thirty-five (70%) studies reported selection criteria beyond elective VH repair, time and place. Only 17 (34%) studies reported a basic list of baseline characteristics meeting our pre-specified criteria (Appendix 3, Online Supplementary Resource 2). Amongst the 34 (68%) studies that did report baseline characteristics (including the 17 studies that met our criteria), 18 (36%) studies showed equivalence between the intervention and comparator groups, whereas 16 (32%) studies reported a difference in one or more baseline characteristics indicating a difference in the group populations. In 16 (32%) studies no comparative analysis of baseline characteristics was performed.
Reported hernia characteristics also varied. Excluding studies that included only primary hernias (8 studies, 16%), the number of prior hernia repairs was only reported in 18 out of 42 (43%) studies. Twenty (40%) studies reported maximal hernia diameter, 12 (48%) prospective and 8 (32%) retrospective. Hernia defect area was reported by 21 studies, again with no detectable difference between the prospective and retrospective studies; 9 (36%) vs. 12 (48%). Thirty-two (64%) studies stated whether hernias were primary, incisional, or both, leaving 18 (36%) that did not state the hernia type included. Only 3 studies [24, 46, 60], graded hernias using either the EHS scale [24, 60] or their own pre-specified scale [46].
Participant recruitment start and finish dates were reasonably reported with 36 (72%) studies reporting both. In contrast, no study reported the end of follow-up date and only 18 (36%) reported the number of deviations from the intended intervention.
Reported outcomes
Hernia recurrence rate was reported in 47 (94%) studies. Three retrospective studies [23, 54, 55] did not report recurrence. However, only 9 (18%) studies defined recurrence; 4 (16%) prospective and 5 (20%) retrospective. None of these studies used the same definition and none referenced a definition of recurrence (Table 2). Two studies [26, 59] reported recurrence but the overall follow-up duration was unclear. Of the remaining 45 studies, recurrence rate, follow-up duration, and detection method varied. Follow-up duration ranged from 3 [47] to 81 months [28], with a median of 27 months. Ten (20%) studies reported a follow-up of between 6 and 12 months. Follow-up duration for the remaining 35 (70%) studies lacked any consistency (Online Supplementary Resource 3). In 21 (42%) studies the follow-up duration differed between treatment arms. Fifteen different methods to detect recurrence were reported across 37 (74%) studies (Online Supplementary Resource 3), ranging from re-operation rate [33] to telephone interview [64]. Seven different detection methods were reported by prospective studies versus 12 different methods for retrospective studies. The most prevalent method used to detect recurrence was clinical assessment followed by a CT scanning if a recurrence was suspected.
Surgical site infection (SSI) was reported by 32 (64%) studies. However, only six (12%) studies, three prospective [19, 28, 60] and three retrospective [29, 53, 57], defined SSI with only three definitions referencing the literature [19, 57, 60]. Two definitions used Center for Disease Control (CDC) wound infection criteria [19, 60], one study referenced NSQIP criteria [57], and the remaining three unreferenced definitions differed [28, 29, 51]. Surgical site infection was reported using an anecdotal grading scale by one study [58]. While one study provided the CDC SSI definition but the results then failed to use this for reporting wound infection rates [19].
Surgical site occurrence (SSO) was reported by four (8%) studies [35, 36, 40, 54]. Only one study [36] defined SSO but without providing a reference. Ten (20%) studies, seven prospective and three retrospective [24, 30, 50], stated patient reported outcomes. Two used the EQ-5D questionnaire [34, 44], one used the French Hernia Club questionnaire [24] and the remaining seven asked ad hoc outcome questions (e.g. time to normal activity, time to return to work). Nine (18%) studies used visual analogue scores to assess pain.
Statistics
Forty-five (90%) studies reported follow-up duration. Multivariable adjusted analysis for hernia recurrence was reported by 10 studies; 7 retrospective and 3 [17, 18, 60] prospective. All 3 prospective studies [17, 18, 60] reported the adjustment factors compared to 5 of 7 for retrospective studies [23, 32, 36, 52, 53]. Eight (16%) studies reported confidence intervals for odds ratios and hazard ratios; 6 [24%] retrospective and 2 (8%) prospective [17, 18]. Only one study [61] reported a complete-case analysis with 100% follow-up at 24 months. No study used imputation to handle missing data so analysis was limited to patients with complete data.
Discussion
In our first methodological systematic review [8], we found that reported variables in randomised controlled trials (RCTs) of VH were heterogenous and lacked standardisation, concluding that clear outcome definitions and a standardised minimum dataset are needed if VH research is to be clinically useful and methodologically credible. Because RCTs are the highest level of evidence [68], we can hypothesise that perioperative variables reported in non-randomised interventional studies of VH repair would be at least as deficient. Therefore, for the present review our emphasis was firmly upon assessment of study methodology. To achieve this, we designed a specific methodological assessment tool using published guidelines [10,11,12,13] (Online Supplementary Material 2).
We found that there was no generally accepted definition of hernia recurrence, no standardised test methods to detect recurrence, no standardised length of follow-up, no universally accepted definition for both surgical site infection (SSI) or surgical site occurrence (SSO), and no standardised evaluation tools for post-operative quality of life and pain. General markers of poor methods included an absence of study protocols and power calculations. This lack of standardisation and methodological vigour limits the validity of published results and, furthermore, impacts upon meta-analytical synthesis.
Perhaps the most pressing issue is a lack of definitions for study outcomes. Historically, the most studied outcomes are surgical site infection (SSI), surgical site occurrence (SSO), and hernia recurrence [69], yet we found researchers defined these items poorly. Regarding hernia recurrence, only 9 (18%) studies defined this and none of these used a standardised definition or referenced the literature. Similarly, methods to detect recurrence and follow-up duration varied. This lack of consensus regarding assessment timing, definitions for recurrence, and test methods used limits the utility of study findings. We advocate using the EHS definition for recurrence [15], ‘a protrusion of the contents of the abdominal cavity or pre-peritoneal fat through a defect in the abdominal wall at the site of a previous repair of an abdominal wall hernia’ as a broad definition for recurrence. However, it is imprecise and an additional definition of recurrence for VH trials that is far more precise and stipulates the exact findings on physical examination and includes the use of imaging to increase accuracy requires development [70]. Indeed, our previous review found that studies employing cross-sectional imaging reported double the hernia recurrence rate than other studies [8]. This supports urgent requirement for standardised detection methods in addition to definitions.
Similarly, we found that SSI and SSO were seldom defined and, even then, rarely referenced standardised definitions form the literature. These findings will not surprise hernia academics since they echo a recent review by Haskins et al. [71], who stated that of the 50 most cited papers describing VH repair, only 9 (18%) used standardised definitions for SSIs and SSOs. Haskins went onto propose definitions for SSI, SSO and SSOPI (surgical site occurrence requiring procedural intervention) that should be adopted by all studies of VH repair. The response from DeBord et al. [72] stated difficulties with the proposal but accepted the need for a “common language”. This editorial concluded by calling for an ‘international task force’ to establish common language for reporting wound complications in the field of abdominal wall reconstruction. We support this.
As well as identifying a paucity for defining outcomes, our methodology review identified additional major reporting deficiencies. No study mentioned writing a protocol, only 2 (4%) performed a power calculation, and only 18 described a primary outcome. These factors are pivotal to good-quality research. Protocols ensure that research is pre-planned and not haphazard, are important for research governance, and demonstrate that authors recognise that ‘quality control needs to be built in from the start rather than the failures being discarded’ at the end [73]. Power calculations are essential; small samples risk type 2 errors whereas too large a sample results in unnecessarily large and costly research, wasting time and effort. Just 18 studies described a primary outcome, an item fundamental to reporting research. In essence, non-randomised interventional studies of VH repair need improved study design and reporting to produce meaningful results.
Surgeons performing such studies should make concerted efforts to reduce bias. We deemed all 50 studies included in this review at high risk of bias. For example, good research practice demands eligibility criteria and keeping a screening log. However, only six studies reported eligibility and when they did so it was implied rather than reported specifically (e.g. ‘57 patients were diagnosed with incisional hernia, 44 underwent surgical repair’ [59]), leaving exclusion criteria in doubt. Poor reporting of ‘eligibility’ exposes studies to concern about potential for selection bias. In general, prospective studies described both the equipment and the intended intervention well and, as a consequence, were at low risk of bias regarding classification of interventions. In contrast, retrospective studies described interventions poorly, suggesting high risk of bias in this category. Retrospective studies cannot control the exact equipment and intervention that was performed on each participant. Studies scored poorly for blinding participant and assessor. While blinding of surgical studies can be difficult, visible skin changes give no clue as to where a mesh was placed or its nature or whether a component separation was performed. Accordingly, blinding should be possible for many hernia studies.
We found that recent publication or higher journal impact factor did not improve quality. This is disappointing because STROBE [13], Newcastle–Ottawa [16], and TIDieR [10] guidelines were published over the time-span of our review, suggesting that hernia researchers are unaware of these recommendations and not party to efforts to improve research quality over the last 20 years [74]. The Ventral Hernia Working Group’s classification of SSO was published in 2010 [69], which we would expect hernia researchers to endorse and use. Systematic reviews of other specialties have demonstrated improved methodology [75] and scoping reviews have shown quality improvement throughout the profession with both publication date and impact factor [76]. As VHs become increasingly prevalent [6], combined with high recurrence rates, these results highlight an urgent need to improve methodology in non-randomised interventional studies of VH repair.
This systematic review has identified a need to construct a standardised minimum dataset for non-randomised VH trials (which greatly outnumber randomised trials). Definition of core variables and outcomes is vital to move the academic hernia community forwards. This endeavour will require international collaboration across academic hernia surgeons. Once achieved, such a minimum dataset will enable trials and registries to report the same peri-operative variables and outcomes, which will facilitate comparisons across them via meta-analysis and multivariate logistic regression, improving our understanding of how each perioperative variable effects outcome. In research generally, there is a worldwide move towards establishing minimum datasets [77, 78]. In this review, and our review of randomised trials [8], we have established evidence that the data collected is currently highly heterogeneous and undefined; clear outcome definitions and a standardised minimum dataset are warranted.
Conclusion
This systematic review is the first methodological review of non-randomised interventional VH studies. The results show that there is a lack of methodological rigour of both prospective and retrospective VH studies. In addition, methodological quality did not improve with publication year or journal impact factor. Studies failed to write protocols prior to implementation, a power calculation was seldom performed, and there was a general lack in defining a primary outcome. Furthermore, studies defined hernia recurrence, surgical site infection and surgical site occurrence poorly and used variable detection methods and grading scales. To solve this, a standardised minimum dataset with a standardised set of peri-operative variables, defined methodology and standardised outcome definitions are needed.
References
Hospital Episode Statistics (2017) UK ventral hernia data: Report on request
Goverment Office for Science (2017) Future of an Ageing Population
NHS Digitial (2017) National statistics: statistics on obesity, physical activity and diet. NHS Digitial, England
Bosanquet DC, Ansell J, Abdelrahman T, Cornish J, Harries R, Stimpson A et al (2015) Systematic review and meta-regression of factors affecting midline Incisional hernia rates: analysis of 14,618 patients. PLoS ONE 10(9):1–18
Faylona JM (2017) Evolution of ventral hernia repair. Asian J Endosc Surg. 10(3):252–258
Parker SG, Reid TH, Boulton R, Wood C, Sanders D, Windsor ACJ (2018) Proposal for a national triage system for the management of ventral hernias. Ann R Coll Surg Engl 100(2):106–110
Köckerling F, Berger D, Jost JO (2014) What is a Certified Hernia Center? The Example of the German Hernia Society and German Society of General and Visceral Surgery. Front Surg. 1(1):26
Parker SG, Wood CPJ, Butterworth JW, Boulton RW, Plumb AAO, Mallett S et al (2018) A systematic methodology review of reported perioperative variables, postoperative outcomes and hernia recurrence from randomised controlled trials of elective ventral hernia repair: clear definitions and standardised datasets are needed. Hernia 22(2):215–226
Agha RA, Barai I, Rajmohan S, Lee S, Anwar MO, Fowler AJ et al (2017) Support for reporting guidelines in surgical journals needs improvement: a systematic review. Int J Surg. 45:14–17
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D et al (2014) Better reporting of interventions: Template for intervention description and replication (TIDieR) checklist and guide. BMJ 7(348):1687
Downs S, Black N (1998) The feasibility of creating a checklist for the assessment of the methodological quality of health care interventions. J Epidemiol Community Heal. 52(6):377–384
Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M et al (2016) ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 12(355):i4919
von Elm E, Altman D, Egger M, Pocock SJ, Gotzsche P, Vandenbroucke JP (2007) Annals of internal medicine academia and clinic the strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting. Ann Intern Med 147(8):573–577
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA et al (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol 62(10):e1–e34
Muysoms F, Campanelli G, Champault GG, DeBeaux AC, Dietz UA, Jeekel J et al (2012) EuraHS: the Development of an international online platform for registration and outcome measurement of ventral abdominal wall Hernia repair. Hernia 16(3):239–250
Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M et al (2008) The Newcastle-Ottawa Scale (NOS) for assessing the quality if nonrandomized studies in meta-analyses. Ottawa Hospital Research Institute, Ottawa
Bingener J, Buck L, Richards M, Michalek J, Schwesinger W, Sirinek K (2007) Long-term outcomes in laparoscopic vs open ventral hernia repair. Arch Surg 142(6):562–567
Greenstein AJ, Nguyen SQ, Buch KE, Chin EH, Weber KJ, Divino CM (2008) Recurrence after laparoscopic ventral hernia repair: a prospective pilot study of suture versus tack fixation. Am Surg 74(3):227–231
Bochicchio GV, De Castro GP, Bochicchio KM, Weeks J, Rodriguez E, Scalea TM (2013) Comparison study of acellular dermal matrices in complicated hernia surgery. J Am Coll Surg 217(4):606–613
Nguyen D, Szomstein S, Ordonez A, Dip F, Rajan M, Menzo E et al (2016) Unidirectional barbed sutures as a novel technique for laparoscopic ventral hernia repair. Surg Endosc. 30(2):764–769
Grønvold LB, Spasojevic M, Nesgaard J-M, Ignjatovic D (2012) Single-incision laparoscopic versus conventional laparoscopic ventral hernia repair: a comparison of short-term surgical results. Surg Laparosc Endosc Percutan Tech. 22(4):354–357
Gonzalez R, Rehnke RD, Ramaswamy A, Smith CD, Clarke JM, Ramshaw BJ (2005) Components separation technique and laparoscopic approach: a review of two evolving strategies for ventral hernia repair. Am Surg 71(7):598–605
Altom LK, Graham LA, Gray SH, Snyder CW, Vick CC, Hawn MT (2012) Outcomes for incisional hernia repair in patients undergoing concomitant surgical procedures. Am Surg 78(2):243–249
Gillion JF, Sanders D, Miserez M, Muysoms F (2016) The economic burden of incisional ventral hernia repair: a multicentric cost analysis. Hernia 20(6):819–830
Ecker BL, Kuo LEY, Simmons KD, Fischer JP, Morris JB, Kelz RR (2016) Laparoscopic versus open ventral hernia repair: longitudinal outcomes and cost analysis using statewide claims data. Surg Endosc 30(3):906–915
Youssef YF, El-Ghannam M (2007) Mesh repair of non-complicated umbilical hernia in ascitic patients with liver cirrhosis. J Egypt Soc Parasitol 37(3 suppl):1189–1197
Stabilini C, Stella M, Frascio M, Salvo L, Fornaro R, Larghero G et al (2009) Mesh versus direct suture for the repair of umbilical and epigastric hernias. Ten-year experience. Ann Ital Chir. 80(3):183–187
Winsnes A, Haapamäki MM, Gunnarsson U, Strigård K (2016) Surgical outcome of mesh and suture repair in primary umbilical hernia: postoperative complications and recurrence. Hernia 20(4):509–516
Al-Salamah SM, Hussain MI, Khalid K, Al-Akeely MH (2006) Suture versus mesh repair for incisional hernia. Saudi Med J 27(5):652–656
Malik AM, Jawaid A, Talpur AH, Laghari AA, Khan A (2008) Mesh versus non-mesh repair of ventral abdominal hernias. J Ayub Med Coll Abbottabad. 20(3):54–56
Jin J, Rosen MJ, Blatnik J, McGee MF, Williams CP, Marks J et al (2007) Use of acellular dermal matrix for complicated ventral hernia repair: does technique affect outcomes? J Am Coll Surg 205(5):654–660
Booth JH, Garvey PB, Baumann DP, Selber JC, Nguyen AT, Clemens MW et al (2013) Primary fascial closure with mesh reinforcement is superior to bridged mesh repair for abdominal wall reconstruction. J Am Coll Surg 217(6):999–1009
Schmidbauer S, Ladurner R, Hallfeldt KK, Mussack T (2005) Heavy-weight versus low-weight polypropylene meshes for open sublay mesh repair of incisional Hernia. Eur J Med Res. 10(6):247–253
Berrevoet F, Maes L, De Baerdemaeker L, Rogiers X, Troisi R, De Hemptinne B (2010) Comparable results with 3-year follow-up for large-pore versus small-pore meshes in open incisional hernia repair. Surgery. 148(5):969–975
Harth KC, Rosen MJ (2010) Endoscopic versus open component separation in complex abdominal wall reconstruction. Am J Surg 199(3):342–347
Azoury SC, Dhanasopon AP, Hui X, De La Cruz C, Tuffaha SH, Sacks JM et al (2014) A single institutional comparison of endoscopic and open abdominal component separation. Surg Endosc 28(12):3349–3358
Moreno-Egea A, Alcaraz AC, Cuervo MC (2013) Surgical options in lumbar hernia laparoscopic versus open repair a long-term prospective study. Surg Innov. 20(4):331–344
Werkgartner G, Cerwenka H, Rappl T, Kniepeiss D, Kornprat P, Iberer F et al (2015) Effectiveness of porcine dermal collagen in giant hernia closure in patients with deleterious fascia constitution after orthotopic liver transplantation. Transpl Int 28(2):156–161
Mitura K, Skolimowska-rzewuska M, Garnysz K (2017) Outcomes of bridging versus mesh augmentation in laparoscopic repair of small and medium midline ventral hernias. Surg Endosc 31(1):382–388
Anadol AZ, Muray A, Kurukahvecioglu O, Tezel E, Ersoy E (2011) Comparison of laparoscopic primary and open primary repair ventral hernias. Surg Laparosc Endosc Percutan Tech. 21(5):301–305
Bogetti P, Boriani F, Gravante G, Milanese A, Ferrando PM, Baglioni E (2012) A retrospective study on mesh repair alone vs. mesh repair plus pedicle flap for large incisional hernias. Eur Rev Med Pharmacol Sci. 16(13):1847–1852
Bencini L, Sanchez LJ, Boffi B, Farsi M, Martini F, Rossi M et al (2009) Comparison of laparoscopic and open repair for primary ventral hernias. Surg Laparosc Endosc Percutan Tech. 19(4):341–344
Solomon TA, Wignesvaran P, Chaudry MA, Tutton MG (2010) A retrospective audit comparing outcomes of open versus laparoscopic repair of umbilical/paraumbilical herniae. Surg Endosc 24(12):3109–3112
Berrevoet F, D’Hont F, Rogiers X, Troisi R, De Hemptinne B (2011) Open intraperitoneal versus retromuscular mesh repair for umbilical hernias less than 3 cm diameter. Am J Surg 201(1):85–90
Moreno-Egea A, Aguayo-Albasini JL, Ballester MM, Cases Baldó MJ (2009) Treatment of incisional hernias adopting an intra-abdominal approach with a new low-density composite prosthetic material: proceed: our preliminary experience on 50 cases. Surg Laparosc Endosc Percutan Tech. 19(6):497–500
Arteaga-Gonzalez I, Martin-Malagon A, Fernandez EM, Carrillo-Pallares A (2010) Which patients benefit most from laparoscopic ventral hernia repair? A comparative study. Surg Laparosc Endosc Percutaneous Tech. 20(6):391–394
Morales-Conde S, Suárez-Artacho G, Socas M, Barranco A (2013) Influence of fibrin sealant in preventing postoperative seroma and normalizing the abdominal wall after laparoscopic repair of ventral hernia. Surg Endosc 27(9):3214–3219
Krpata DM, Blatnik JA, Novitsky YW, Rosen MJ (2012) Posterior and open anterior components separations: a comparative analysis. Am J Surg 203(3):318–322
Colon MJ, Telem DA, Chin E, Weber K, Divino CM, Nguyen SQ (2011) Polyester composite versus PTFE in laparoscopic ventral hernia repair. JSLS. 15(3):305–308
Kitamura RK, Choi J, Lynn E, Divino CM (2013) Suture versus tack fixation of mesh in laparoscopic umbilical hernia repair. JSLS. 17(4):560–564
Azoury SC, Dhanasopon AP, Hui X, Tuffaha SH, De La Cruz C, Liao C et al (2014) Endoscopic component separation for laparoscopic and open ventral hernia repair: a single institutional comparison of outcomes and review of the technique. Hernia 18(5):637–645
Iacco A, Adeyemo A, Riggs T, Janczyk R (2014) Single institutional experience using biological mesh for abdominal wall reconstruction. Am J Surgery. 208(3):480–484
Froylich D, Segal M, Weinstein A, Hatib K, Shiloni E, Hazzan D (2016) Laparoscopic versus open ventral hernia repair in obese patients: a long-term follow-up. Surg Endosc 30(2):670–675
Warren JA, Cobb WS, Ewing JA, Carbonell AM (2017) Standard laparoscopic versus robotic retromuscular ventral hernia repair. Surg Endosc 31(1):324–332
Earle D, Seymour N, Fellinger E, Perez A (2006) Laparoscopic versus open incisional hernia repair: a single-institution analysis of hospital resource utilization for 884 consecutive cases. Surg Endosc 20(1):71–75
Singhal V, Szeto P, VanderMeer TJ, Cagir B. Ventral hernia repair: outcomes change with long-term follow-up. JSLS. 2012 Jul-Sep;16(3):373–9
Ballem N, Parikh R, Berber E, Siperstein A (2008) Laparoscopic versus open ventral hernia repairs: 5 year recurrence rates. Surg Endosc 22(9):1935–1940
Harth KC, Blatnik JA, Rosen MJ (2011) Optimum repair for massive ventral hernias in the morbidly obese patientis panniculectomy helpful? Am J Surg 201(3):396–400
Kurmann A, Beldi G, Vorburger SA, Seiler CA, Candinas D (2010) Laparoscopic incisional hernia repair is feasible and safe after liver transplantation. Surg Endosc 24(6):1451–1455
Kurmann A, Visth E, Candinas D, Beldi G (2011) Long-term follow-up of open and laparoscopic repair of large incisional hernias. World J Surg 35(2):297–301
Godara R, Garg P, Raj H, Sham S (2006) Comparative evaluation of “sublay” versus “onlay” meshplasty in ventral hernias. Indian J Gastroenterol. 25(4):222–223
Qadri SJF, Khan M, Wani SN, Nazir SS, Rather A (2010) Laparoscopic and open incisional hernia repair using polypropylene mesh—a comparative single centre study. Int J Surg. 8(6):479–483
Schroeder AD, Debus ES, Schroeder M, Reinpold WMJ (2013) Laparoscopic transperitoneal sublay mesh repair: a new technique for the cure of ventral and incisional hernias. Surg Endosc 27(2):648–654
Olmi S, Magnone S, Erba L, Bertolini A, Croce E (2005) Results of laparoscopic versus open abdominal and incisional hernia repair. JSLS. 9(2):189–195
Lomanto D, Iyer SG, Shabbir A, Cheah WK (2006) Laparoscopic versus open ventral hernia mesh repair: a prospective study. Surg Endosc 20(7):1030–1035
Stojiljkovic D, Kovacevic P, Visnjic M, Jankovic I, Stevanovic G, Stojiljkovic P et al (2013) Comparative analysis of autodermal graft and polypropylene mesh use in large incisional hernia defects reconstruction. Vojn Pregl. 70(2):182–188
Scheuerlein H, Rauchfuss F, Gharbi A, Heise M, Settmacher U (2011) Laparoscopic incisional hernia repair after solid-organ transplantation. Transplant Proc. 43(5):1783–1789
OCEBM Levels of Evidence Working Group (2016) The oxford levels of evidence 2. Oxford cent evidence-based med
Breuing K, Butler CE, Ferzoco S, Franz M, Hultman CS, Kilbridge JF et al (2010) Incisional ventral hernias: review of the literature and recommendations regarding the grading and technique of repair. Surgery. 148(3):544–558
Halligan S, Parker SG, Plumb AAO, Boulton RW, Mallett S (2018) Use of imaging for pre- and post-operative characterisation of ventral hernia: systematic review. Br J Radiol 91(1089):20170954
Haskins IN, Horne CM, Krpata DM, Prabhu AS, Tastaldi L, Perez AJ et al (2018) A call for standardization of wound events reporting following ventral hernia repair. Hernia 22(5):729–736
DeBord J, Novitsky Y, Fitzgibbons R, Miserez M, Montgomery A (2018) SSI, SSO, SSE, SSOPI: the elusive language of complications in hernia surgery. Hernia 22(5):737–738
Altman DG (1994) The scandal of poor medical research. BMJ 308(6924):283–284
Moher D, Altman DG, Schulz KF, Elbourne DR (2004) Opportunities and challenges for improving the quality of reporting clinical research: CONSORT and beyond. CMAJ 171(4):349–350
Péron J, Pond GR, Gan HK, Chen EX, Almufti R, Maillet D et al (2012) Quality of reporting of modern randomized controlled trials in medical oncology: a systematic review. J Natl Cancer Inst 104(13):982–989
Samaan Z, Mbuagbaw L, Kosa D, Borg Debono V, Dillenburg R, Zhang S et al (2013) A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc. 6(6):169–188
Coulman KD, Hopkins J, Brookes ST et al (2016) A core outcome set for the benefits and adverse events of bariatric and metabolic surgery: the BARIACT project. PLoS Med 13(11):e1002187
Haywood KL, Whitehead L, Perkins GD (2019) An international, consensus-derived core outcome set for cardiac arrest effectiveness trials: the COSCA initiative. Curr Opin Crit Care 25(3):226–233
Funding
This work was funded by the UK National Institute for Health Research (Grant RfPB PB-PG-0816-20005) and Allergan PLC. Neither funders have been involved in the planning, methodology, analysis or write up of the research. National Institute of Health Research, Room 132, Richmond House, 79 Whitehall, London, SW1A 2NS. Allergan Plc, Clonshaugh Business and Technology Park, Coolock, Dublin, D17 E400, Ireland.
Mallett S is supported by NIHR Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust and the University of Birmingham.
This report presents independent research supported by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, or the Department of Health.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Windsor A.C.J declares conflicts of interest not directly related to the submitted work; consultant advisor for TELA BIO; educational grants and speaker for: BARD Davol Inc, Allergan Plc and Cook. Parker S.G, Halligan S, Erotocritou M, Wood C.P.J, Boulton R.W, Plumb A.A.O, and Mallett S declare no conflict of interest.
Ethical approval
Ethical permission is not required by our centre for systematic reviews of available primary literature.
Human and animal participants
This article does not contain any studies with human participants performed by any of the authors.
Informed consent
This article does not include patients, and therefore informed consent was not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Parker, S.G., Halligan, S., Erotocritou, M. et al. A systematic methodological review of non-randomised interventional studies of elective ventral hernia repair: clear definitions and a standardised minimum dataset are needed. Hernia 23, 859–872 (2019). https://doi.org/10.1007/s10029-019-01979-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10029-019-01979-9