Abstract
Taxonomies are classification systems that help researchers conceptualize phenomena based on their dimensions and characteristics. To address the problem of ‘ad-hoc’ taxonomy building, Nickerson et al. (2013) proposed a rigorous taxonomy development method for information systems researchers. Eight years on, however, the status quo of taxonomy research shows that the application of this method lacks consistency and transparency and that further guidance on taxonomy evaluation is needed. To fill these gaps, this study (1) advances existing methodological guidance and (2) extends this guidance with regards to taxonomy evaluation. Informed by insights gained from an analysis of 164 taxonomy articles published in information systems outlets, this study presents an extended taxonomy design process together with 26 operational taxonomy design recommendations. Representing an update for taxonomy designers, it contributes to the prescriptive knowledge on taxonomy design and seeks to augment both rigorous taxonomy building and evaluation.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
“We categorize as we do because we have the brains and bodies we have and because we interact in the world as we do.” (Lakoff and Johnson 1999, p. 18).
The categorization of objects, also known as classification, is a fundamental cognitive process. Thereby, humans classify objects to simplify their understanding of the world and avoid being overwhelmed by the sheer diversity of objects (Lakoff 1987; De Langhe and Fernbach 2019). Taxonomies help humans classify objects according to similarities and differences, and thus enable researchers and practitioners to describe, understand, and analyze phenomena of interest (Nickerson et al. 2013). Taxonomies have a long history in the natural and social sciences (Bailey 1994; Eldredge and Cracraft 1980; Sokal and Sneath 1963) and play a central role in the information systems (IS) discipline, especially given the speed of socio-technical progress that requires continuous efforts of understanding.
To support IS researchers in rigorously designing taxonomies, Nickerson et al. (2013) proposed a method for developing taxonomies in a systematic, transparent and replicable manner. Since then, their method has been widely applied, as evidenced by the fact that since 2013 about two-thirds of the taxonomies published in IS outlets – and analyzed in this study – follow Nickerson et al.’s (2013) method. Examples stem from domains as diverse as mobile health (Varshney 2014), digital finance (Gomber et al. 2017), and virtual assistance (Janssen et al. 2020).
Apart from being artefacts for describing and classifying phenomena, taxonomies have gained attention in IS because they can also serve as a foundation for sense-making (Gregor and Hevner 2013) and theory building (Doty and Glick 1994). According to Gregor's (2006) theory types, taxonomies can be theories for analyzing, which is the most basic form of theory (so-called taxonomic theory, Varshney et al. 2017). Taxonomic theories can be used for the development of other theories (e.g., explanatory, predictive or design theories) by constituting their fundamental constructs and relationships (Gregor 2006; Varshney et al. 2015; Whetten 1989).
Despite the growing interest in taxonomies, as reflected in the increasing number of taxonomies published, IS researchers face two related challenges: Taxonomy building and taxonomy evaluation.Footnote 1 With regard to taxonomy building, we observed that taxonomy articles adopt existing methods inconsistently and often report the taxonomy building in a non-transparent manner. At the same time researchers ask for more guidance on how to implement the individual steps of existing taxonomy development methods (e.g., Mwilu et al. 2015). Regarding evaluation, we found that taxonomies are rarely evaluated and researchers call for further support on how to evaluate them (e.g., Kazan et al. 2018; Krieger and Drews 2018). From these observations, we conclude that taxonomy designers would highly benefit from additional guidance supporting a rigorous design of taxonomies. A need that has also been emphasized by the taxonomy experts whom we interviewed for this study. They found themselves often asking questions such as “what are good practices to build my taxonomy?” or “how shall I evaluate my taxonomy?”. Against this backdrop, we address the following research question: How should taxonomies be built and evaluated?
Based on the systematic analysis of how taxonomies have been built and evaluated in IS research since the publication of Nickerson et al.’s (2013) method, we provide an update for taxonomy designers for methodologically augmenting taxonomy design. This update is anchored in the design science research (DSR) paradigm and takes into account previous efforts to advance taxonomy design in the IS discipline and beyond. As with the original method, also our advanced and extended guidance on taxonomy design is not limited to the IS discipline. Instead, IS-specific peculiarities arise from the phenomena that the IS discipline is typically concerned with. Such phenomena are changing and emerging at an unprecedented speed and scale. Moreover, they are complex and require constant consideration of which (parts of) phenomena are “entirely new and unique, a significant variation […], or just a retread of what we already have” (Nickerson et al. 2013, p. 337). Given that digital transformation often drives the change and the emergence of phenomena, there is a continuous need for coordination between research and practice. For researchers in the IS discipline to pick up on such developments, the identification and structuring of characteristics and dimensions using taxonomies as well as the transparent reporting of their taxonomy design are important.
Our main contribution consists of two parts: First, we advance existing methodological guidance on taxonomy building. Second, we extend this guidance regarding the evaluation of taxonomies. As result, we present an extended taxonomy design process (ETDP) together with 26 operational taxonomy design recommendations (TDR). In sum, we seek to refine and complement the seminal work of Nickerson et al. (2013), offering prescriptive guidance that facilitates a more rigorous building and evaluating of taxonomies.
2 Research Background
2.1 Taxonomy Development in Information Systems and Adjacent Fields
Taxonomies are fundamental in research and practice, which is why taxonomy design has been investigated in various disciplines such as natural sciences (Sokal and Sneath 1963), social sciences (Bailey 1994), organizational science (McKelvey 1975), and strategic management (Hambrick 1984). As a well-accepted method in terms of citations, Bailey (1994) provided a sound overview of and guidance on how to develop taxonomies in social science. Bailey differentiates between the conceptual (i.e., deducing taxonomic structure from a theoretical foundation), the empirical (i.e., grouping inductively via statistical methods), and the operational (i.e., mapping both conceptual and empirical levels) approach. In computer science, Bayona-Oré et al. (2014) proposed a method with 24 activities along five phases (i.e., planning, identification and extraction of information, design and construction, testing and validation, and deployment) to develop taxonomies in software engineering. Thereby, they highlighted demands for clearly formulating a user problem that should be addressed by means of a taxonomy as well as for testing and using a taxonomy to obtain insights concerning its practical usefulness. Usman et al. (2017) refined Bayona-Oré et al. (2014) method and proposed 13 activities ranging from the selection of a classification procedure type to the identification of top-level dimensions and validation through benchmarking. Next to these general activities, other researchers present more specific guidelines for testing taxonomies with conceptual and analytical models (Doty and Glick 1994) or for typical taxonomy building issues (Hambrick 1984). Additional guidance focuses on fields such as requirements-driven taxonomies (Notheisen et al. 2019) or business reporting taxonomies (Ojala et al. 2018). A detailed overview of related guidance on taxonomy design can be found in Appendix 1 (available online via http://springerlink.fh-diploma.de).
While other disciplines extensively reflected on taxonomy design, taxonomies in IS research have often been built rather intuitively, for instance by applying methods such as clustering (Posey et al. 2017), qualitative content analysis (Goo et al. 2000), or deductive reasoning (Al-Debei and Avison 2010). Nickerson et al. (2013) provided the first – and so far the only – well-conceived taxonomy development method for the IS discipline. This is important as IS research is concerned with emerging and rapidly evolving technologies, and thus needs a foundation for understanding and analyzing socio-technical phenomena. Nickerson et al. (2013) method comprises seven steps, combining both an inductive and a deductive approach to build taxonomies iteratively. It starts with the determination of a meta-characteristic that is derived from the purpose and target users of the taxonomy. Next, the objective (i.e., taxonomy is valid) and subjective (i.e., taxonomy is useful) ending conditions help to determine when the iterative method can be terminated. Afterward, researchers have to decide on the approach to be adopted for the first/next iteration. If data about real-world objects are available, researchers are advised to follow an inductive approach (i.e., empirical-to-conceptual). If a significant understanding of the phenomenon in focus already exists, a deductive approach (i.e., conceptual-to-empirical) should be employed. The taxonomy development process continues with the next iteration until all ending conditions are met (find the original method in Appendix 1).
Despite being the de-facto standard in the IS discipline, some researchers have adapted Nickerson et al. (2013) method. For instance, Mwilu et al. (2015) argued that the ending conditions are “often difficult to apply in practice” (p. 3) and highlighted the need for advancing steps such as the specification of the object sample. In another example, Land et al. (2013) have extended the original method by introducing a library of case studies (i.e., real-life objects) and a testing phase to validate a taxonomy version using (new) cases and stakeholder feedback. In this vein, Sarkintudu et al. (2018) have provided an extension that seeks to refine a taxonomy based on insights gained during the taxonomy’s usage. In some instances, researchers have explicitly raised the need for specific further guidance regarding taxonomy evaluation (e.g., Kazan et al. 2018; Krieger and Drews 2018), which is apparent in statements such as “the main development cycle [for taxonomies] does not include the evaluation step, leaving it open for researchers” (Chasin et al. 2018, p. 299).
Even though first attempts have started to advance and complement existing methodological guidance, Nickerson et al.’s (2013) method is by far the most referenced method in IS that has been served as a blueprint for numerous of taxonomy projects (see status quo in Sect. 4). Moreover, this method is grounded in the literature on taxonomy design in IS and adjacent fields (see seminal work such as Bailey 1994; Doty and Glick 1994) and serves as a starting point for refined and adapted methodological guidance (e.g., Land et al. 2013; Sarkintudu et al. 2018). Against this backdrop, we have decided to build this study primarily upon the seminal work of Nickerson et al. (2013), while considering additional guidance also beyond IS research to advance and extend the taxonomy design process.
2.2 Taxonomies as Design Science Research Artefacts
Taxonomies help to identify and structure characteristics and dimensions towards describing, understanding, and analyzing phenomena. This applies to both, DSR and non-DSR (such as behavioral science that seeks to develop and test theories, Doty and Glick 1994). This study builds upon previous work by Nickerson and colleagues positioning taxonomies and their design in the DSR paradigm (Nickerson et al. 2013). Given that DSR has the potential to contribute “both practical relevance (via its emphasis on useful artefacts) and scientific rigor (via the formulation of design theories)” (Baskerville et al. 2018, p. 358) this study relies on DSR for the advancement and extension of methodological guidance for taxonomy design.
DSR contributions come in forms as diverse as design artefacts, design theories, and design processes (Baskerville et al. 2018). We see taxonomies as artefacts and consider them – in contrast to real-world phenomena – as artificial objects that solve practical problems (Simon 1996; Hevner et al. 2004) (here identifying and structuring dimensions and characteristics of phenomena). In accordance with the DSR paradigm, this study seeks to provide useful artefacts and not, as common in non-DSR research, to achieve truth or truthlikeness (Hevner and Chatterjee 2010; Goldkuhl 2004) depending on the underlying epistemological and ontological assumptions (Baskerville et al. 2015; Niehaves 2007; Frank 2006).
Generally, four basic artefact types are differentiated in DSR (March and Smith 1995), namely constructs (i.e., concepts and vocabulary of a domain), models (i.e., constructs and their relationship to represent a phenomenon), methods (i.e., processes to perform tasks and achieve goals), and instantiations (i.e., situated implementations that operationalize constructs, models, and methods). Following previous research examining DSR artefacts (e.g., Offermann et al. 2010; Sangupamba et al. 2014), we consider taxonomies as structure-giving artefacts in the form of models for four reasons: First, models “can be viewed simply as a description that is a representation of how things are” (March and Smith 1995, p. 256), which also applies to taxonomies that capture what things are out there (e.g., Iivari 2007). Second, models provide “certain concepts [of a domain] and relationships among them” (March and Smith 1995, p. 256) for understanding phenomena. Taxonomies help researchers to organize knowledge by representing relevant dimensions and corresponding characteristics (i.e., constructs) and thereby reflect relationships among dimensions and characteristics. These relationships are visualized, for instance, through hierarchical structures (e.g., Prat et al. 2015) or multi-layer structures encompassing abstract layers, dimensions, and characteristics (e.g., Janssen et al. 2020). Third, models share three essential properties (borrowed from general model theory as presented by Stachowiak 1973) that are also addressed by taxonomies: Representation – taxonomies represent existing or future, and natural or artificial objects; reduction – taxonomies capture not all attributes of phenomena and instead focus on those attributes that are relevant to a taxonomy’s purpose and target user group; and pragmatism – taxonomies can fulfil various purposes such as describing or analyzing phenomena. Fourth, Nickerson et al. (2013) and other IS taxonomy designers have already positioned their research outcomes as models (e.g., Oberländer et al. 2018; Yang and Varshney 2017).
Positioning taxonomies as models allows taxonomy designers to draw from the rich body of DSR knowledge. As a consequence, the DSR knowledge provides foundations and methodological guidance that help researchers to motivate an artefact’s objective (Peffers et al. 2007) and thereby anchor artefacts in the problem and solution space (vom Brocke et al. 2020). DSR generally differentiates two main activities for building and evaluating artefacts, commonly referred to as the build-evaluate pattern (e.g., Hevner et al. 2004; March and Smith 1995; Sonnenberg and vom Brocke 2012). A need for evaluation arises not only from DSR in general (e.g., Prat et al. 2015; Venable et al. 2016) but also from the application of taxonomies in particular, as taxonomies need to be useful for certain goals such understanding phenomena or making decisions (e.g., De Langhe and Fernbach 2019; Morana et al. 2020). Transferring the build-evaluate pattern to taxonomy design, we observed that further guidance with regards to the evaluation of taxonomies would be helpful, which has been emphasized by Nickerson et al. (2013) who stated that “the resulting taxonomy needs to be evaluated for its usefulness” (p. 346). Rigorously built and evaluated artefacts allow researchers to generalize knowledge (Baskerville and Pries Heje 2019). This is an important aim of DSR projects − including those that intend to design taxonomies – to add design knowledge to the DSR knowledge base (Hevner et al. 2004; Niehaves 2007). Depending on the scope and size, the design of a taxonomy can be a stand-alone DSR project or part of a larger DSR project. Moreover, anchoring taxonomy design under the DSR paradigm allows us to promote the iterative nature of artefact design as well as to organize the design process (ETDP) and the design recommendations (TDR) in a structured way, for example, following the steps of the DSR methodology proposed by Peffers et al. (2007) (see Sect. 5).
3 Research Method
In this study, we aim to answer the research question of how taxonomies should be built and evaluated. To this end, we first assess relevant articles building and/or evaluating a taxonomy. Given the insights (e.g., good practices and challenges) from this status quo analysis, we then advance existing methodological guidance and extend this guidance with regards to taxonomy evaluation.
3.1 Status Quo: Data Collection and Coding
To assess the status quo of taxonomy design in IS, we followed the recommendations of Templier and Paré (2018) in a two-phased approach. In the first phase, we aimed at gaining insights into the operationalization of taxonomy design, presentation and evaluation. Given this aim, we searched and screened relevant literature (see Table 1). In the second phase, we analyzed our sample of identified taxonomy articles and extracted relevant data (using coding attributes such as taxonomy design method, presentation form, evaluation method and criteria). On this empirical basis, we synthesized findings and identified ‘good practices’ as transparent and comprehensible operationalizations of taxonomy design steps. We present a summary of our coding results in Table 2 and methodological details on the status quo analysis in Appendix 2.
3.2 Extended Taxonomy Design Process and Design Recommendations
For deriving the ETDP and the TDR, we particularly drew on examples of transparent and comprehensible taxonomy design, adherence to good practice, and methodological gaps collected during the coding of the sample. Following inductive reasoning (e.g., Hempel 1966), we critically discussed the collected codes (i.e., attributes such as good practices and challenges) to select the most potent attributes in terms of guiding future taxonomy designers. Based on this selection, we advanced and extended the steps of the method proposed by Nickerson et al. (2013) to create the ETDP, formulated TDR, and assigned these recommendations to the steps of the ETDP. The ETDP and the TDR were iteratively (re-)formulated by the author team. Specifically, with regards to further guidance on the evaluation of taxonomies (particularly evaluation goals and ex post evaluation), we built on our sample of taxonomy articles that evaluate taxonomies (56 unique articles) following analytical and abductive reasoning principles (e.g., Gregory and Muntermann 2011; Van de Ven and Johnson 2006). Further, we anchor our guidance in well-established DSR knowledge on artefact evaluation, among others, following the structuring questions of ‘why’, ‘how’, and ‘what’ to evaluate (e.g., March and Smith 1995; Prat et al. 2015; Venable et al. 2016) (see Table 6).
For the evaluation of the ETDP and the TDR, we conducted semi-structured expert interviews (Myers and Newman 2007) that can be used “to confirm what is already known whilst at the same time providing the opportunity for learning” (Recker 2013, p. 91). Our evaluation's goal was to verify the understandability and expected usefulness of the ETDP and the TDR. We first tested our interview procedure and conducted four pre-tests with IS researchers who have already published at least one taxonomy in the conference proceedings of ICIS or ECIS. Once the interview procedure was robust, we started to recruit experts (i.e., authors who have published taxonomy articles in journals listed in the AIS Senior Scholars’ Basket). The participants were provided with the ETDP and the TDR beforehand. Each interview consisted of four parts, i.e., (1) motivation and problem awareness related to taxonomy design, discussions on (2) the ETDP and (3) the TDR, and (4) additional feedback (e.g., assessment of understandability and usefulness). During the interview, one co-author took the role of the interviewer. At least one other co-author was responsible for asking clarifying questions and documenting the interviewee’s responses and questions in the form of a protocol. We analyzed the feedback and revised both the ETDP and the TDR (see Appendix 4 for details on the taxonomy experts and their feedback).
Moreover, to validate our result’s coverage and ensure alignment with the existing body of knowledge, we cross-checked the ETDP and TDR with additional taxonomy design guidance in and beyond IS (see Appendix 1 for a detailed overview of related guidance).
4 Status Quo: Taxonomy Design in IS
The analysis of taxonomy articles revealed two major insights: First, taxonomy building (i.e., research process and product) often remains unclear to readers and, second, a substantial part of taxonomy projects refrains from evaluating the taxonomies. In Table 2, we summarize the data relevant to illustrate our two major insights, while Appendix 2 presents a detailed overview of the descriptive statistics.
Regarding taxonomy building, we found 160 articles in which researchers built a taxonomy. Despite the fact that in 77% of these articles the taxonomies are built with reference to the method of Nickerson et al. (2013), only 22% provide a transparent description of their taxonomy building process.Footnote 2 Even among the articles citing Nickerson et al. (2013), a substantial proportion does not report on elementary design decisions (e.g., no dedicated method section). Such lack of transparency and consistency may complicate the replication of the research process, weaken credibility, limit robustness, and hinder future extensions as well as the taxonomies’ adoption. Further, about two-thirds of the articles present their taxonomy only vaguely.Footnote 3 If a taxonomy is insufficiently presented, it is more difficult to assess its quality, use it for the classification of objects, and extend or revise it in future research.
Regarding taxonomy evaluation, we found that taxonomies are evaluated in around one-third of all analyzed articles (see Table 1). In these articles, we observed ten different types of evaluation methods (see Appendix 2). The most commonly used method (58%) involves the illustration of a scenario with real-world objects (e.g., description and/or classification of real-world objects using a taxonomy) and is followed by the illustration with existing research (15%) and expert interviews (9%). Further, we found 41 different evaluation criteria spearheaded by usefulness (59%), comprehensiveness (47%), and applicability (39%). There are a few predominant evaluation criteria and many with low-frequency use (66% of the evaluation criteria are mentioned in three or fewer taxonomy articles; see Appendix 2).
In summary, our analysis of taxonomy articles reveals that existing methodology for taxonomy building is cited by a large proportion of articles. However, this guidance is adopted inconsistently, the process of taxonomy building often remains unclear (i.e., relevant design decisions in the research process are only partially reported), and taxonomies are hardly evaluated. We conclude that some of these issues may be attributable to operationalization problems, where researchers ask for more guidance on how to implement the individual steps of existing methods (e.g., Mwilu et al. 2015). For instance, Nickerson et al. (2013) provide support for the ex-ante evaluation of taxonomies (i.e., during the development) in the form of objective and subjective ending conditions (Nickerson et al. 2013; Szopinski 2019b). However, the authors do not provide detailed guidance with regards to an ex post evaluation (i.e., after the development) stating that they “are not able [..] to give sufficient conditions other than to say that a taxonomy is useful if others use it” (Nickerson et al. 2013, p. 342). From these insights, we infer a demand for further guidance with regards to the advancement and extension of the existing methodology. More specifically, we argue that an extended taxonomy design process with design recommendations for taxonomy building and ex post evaluation is needed.
5 Results
In response to the outlined demands for further methodological guidance for the design of taxonomies, we developed the ETDP (see Fig. 1) and the TDR (see Table 3). Both, the ETDP and the TDR reflect the results from the systematic analysis of taxonomy articles and include refinements from interviews with experts who provided feedback on the ETDP’s and TDR’s understandability as well as expected usefulness (see Appendix 4 for details on the taxonomy experts and their feedback). Together, the ETDP and the TDR are intended to guide researchers through the entire taxonomy design by stimulating critical reflection and careful design decisions − while building, evaluating, and communicating taxonomies. We organize the ETDP and the TDR along the six activities of the DSR methodology proposed by Peffers et al. (2007). The starting point is the taxonomy development method by Nickerson et al. (2013).
Like the original method and in line with the DSR paradigm, the ETDP is iterative. As a result, specific entry and exit points for design iterations are provided. This is important because it is not possible to determine a priori when the taxonomy will be completed, i.e. after how many iterations. More specifically, there are three steps after which taxonomy designers can return to previous steps (exit points after Step 12, 14, and 17, see beginning of dashed arrows in Fig. 1). From any of these three steps, researchers can return to one of the following four steps (entry points before Step 2, 4, 6, and 10, see end of dashed arrows in Fig. 1). In this way, the ETDP enables researchers to consciously distribute steps across research cycles (e.g., building and evaluating a taxonomy separately) allowing them to use the ETDP for the evaluation of taxonomies that have not been evaluated yet. This iterative nature of the ETDP enables taxonomy designers to update, extend, and refine existing taxonomies over time without the need to repeat the entire taxonomy design process.
Building upon and complementing the ETDP, 26 TDR provide researchers with operational support and examples of good practice. In other words, the TDR may guide taxonomy designers by encouraging and enabling them to apply the ETDP in a specific research context. This context is shaped by the phenomenon under consideration, the taxonomy’s target user group(s) and its purpose(s).
5.1 Identify Problem and Motivate
Steps 1 to 3 of the ETDP anchor the taxonomy in the DSR problem and solution space (vom Brocke et al. 2020). Thereby, they support explicating the specific problem that is to be solved using the taxonomy to be designed in a given context. This involves specifying the observed phenomenon (Step 1), the taxonomy’s target user group(s) (Step 2), and the intended purpose(s) (Step 3). Justifying the value of the proposed solution (e.g., Hevner et al. 2004; vom Brocke et al. 2020; vom Brocke and Maedche 2019) helps not only to understand why a taxonomy is important for its intended user group(s), but also allows to emphasize the importance of the phenomenon under consideration. Many researchers face the problem that the constructs underlying a phenomenon are either unknown or known but unstructured. This in turn leads to two typical purposes of taxonomies:
Purely structuring purpose: The purpose of the taxonomy is to structure known constructs and their relationships related to the phenomenon under consideration (e.g., established phenomena for which constructs must be synthesized from different contexts or disciplines).
Combined identification and structuring purpose: The purpose of the taxonomy is to identify and structure constructs and their relationships for the phenomenon under consideration (e.g., emerging phenomena for which constructs must be identified for the first time).
There is an (re-)entry point before Step 2, because later demonstration or evaluation of the taxonomy may reveal that it is only be suitable for parts of the originally defined target group(s) (i.e., specification) or even beyond the originally defined target group(s) suitable (i.e., generalization).
TDR 1 to 3 help clarifying how a taxonomy can be a solution to the problem of an unknown/unstructured phenomenon and how such a taxonomy relates to potentially already existing taxonomies.
5.2 Define Objectives of a Solution
Steps 4 and 5 of the ETDP support reflecting upon the taxonomy’s objectives. So far, researchers have followed Nickerson et al. (2013) and determined both the meta-characteristic and the ending conditions. Beyond that, we suggest determining evaluation goal(s) before initiating the taxonomy design to align the problem and the solution space of a taxonomy project at an early stage (e.g., McKay et al. 2012; vom Brocke et al. 2020; vom Brocke and Maedche 2019). This requires not only deciding which angle a taxonomy takes on the phenomenon under consideration (determining meta-characteristic, Step 4), but also when taxonomy building is completed (determining ending conditions) and how to evaluate taxonomies (determining evaluation goal(s), Step 5). This is because the definition of a future solution includes setting the goals not only of the artefact’s building but also of its evaluation (Sonnenberg and vom Brocke 2012). We derived potential evaluation goals from our sample of taxonomy articles following analytical and abductive reasoning principles (e.g., Gregory and Muntermann 2011; Van de Ven and Johnson 2006) (see Sect. 3.2). The evaluation goals comprise, for example, better describing, identifying, classifying, analyzing, and clustering of objects that represent a certain phenomenon compared to doing so without a taxonomy or other classification schemes (see Table 4).
In line with defining the problem space in DSR, the meta-characteristic defines what is relevant for the specific taxonomy design and what is not. Consequently, all the following characteristics and dimensions of the taxonomy must relate to this meta-characteristic. A problem-space chosen in this way must always be considered in its environment (Simon 1996). A taxonomy’s environment is shaped by the target user group(s), purpose(s) and evaluation goal(s) which in turn should be aligned to each other right from the beginning of the taxonomy design process. Therefore, we stress the importance of defining the meta-characteristic, bearing in mind that it can be further refined after the first iteration(s), as it is impossible to robustly determine it at this stage – even more so in the case of emerging phenomena. Further, there is no ‘right’ meta-characteristic since phenomena can be viewed from different angles. Rather it is important to explicate the specific angle chosen. To allow refining the meta-characteristic during taxonomy design, the ETDP provides a (re-)entry point before Step 4.
TDR 4 to 6 help determine and refine the meta-characteristic that defines the objective of the taxonomy design as the specific angle from which the phenomenon is perceived (see Fig. 2).
5.3 Design and Development
Steps 6 to 10 of the ETDP support building the taxonomy. We build on the corresponding steps as proposed by Nickerson et al. (2013) and carefully refine a selected part. In this way, we emphasize the idea of combining inductive and deductive approaches for designing taxonomies (Steps 6 to 9). As the creation/revision of taxonomies is often not made explicit, we constitute a new, separate step in which we offer researchers pre-defined taxonomy operations (Step 10). By defining a taxonomy operation as a concrete change made to a taxonomy following an iteration, we thus allow for a taxonomy being altered without the need to re-examine existing or examine new objects (e.g., through renaming or re-structuring characteristics and/or dimensions after the evaluation). Taxonomy operations can change either one or more characteristics and/or one or more (sub-) dimensions. We combine previous work on taxonomy operations (Mwilu et al. 2015) and ‘CRUD functions’ from computer science (Martin 1983) to conceptualize a set of taxonomy operations which includes adding, updating, and deleting characteristics and dimensions (see Table 5). These operations support creating, revising, and documenting taxonomies in a more consistent and transparent way. Since taxonomy design is iterative without knowing a priori the kind and number of iterations as well as whether objects need to be investigated or not, the ETDP provides (re-)entry points before Step 6 and 10.
TDR 7 to 16 help taxonomy designers to decide on the number of iterations required for taxonomy building, which approach to adopt, and when (i.e., conceptual-to-empirical or empirical-to-conceptual) as well as how to implement the individual iterations for iteratively creating and revising the taxonomy.
5.4 Demonstration
Steps 11 and 12 of the ETDP support demonstrating that a taxonomy is formally valid and satisfies the definition of a taxonomy independently of its purpose(s) and target user group(s). The objective ending conditions proposed by Nickerson et al. (2013) help to demonstrate whether a taxonomy meets the essential criteria for a taxonomy. This involves objectively verifying whether a taxonomy is mutually exclusive and collectively exhaustive. TDR 17 to 19 help demonstrating the validity of the taxonomy.
5.5 Evaluation
Steps 13 and 14 of the ETDP support initiating the evaluation of a taxonomy while considering the taxonomy’s purpose(s) and targeted user group(s). Here, researchers have to evaluate whether the taxonomy is applicable and useful for providing structure to the phenomenon under consideration. The subjective ending conditions proposed by Nickerson et al. (2013) collectively constitute the necessary condition of an ex ante evaluation during the taxonomy building iterations (Nickerson et al. 2013; Szopinski et al. 2019b). This comes down to subjectively verifying that a taxonomy is concise, robust, comprehensive, extendible, and explanatory. Our status quo analysis shows that subjective ending conditions are often reviewed by reaching a consensus among the co-authors of an article as to whether the subjective ending conditions have been met (i.e., whether the taxonomy is perceived applicable). Apart from this being a necessary condition, the rigorous evaluation of a taxonomy requires that it also meets sufficient conditions after the building process has been terminated (i.e., ex post evaluation) (Nickerson et al. 2013; Prat et al. 2015; Sonnenberg and vom Brocke 2012).
Steps 15 and 16 of the ETDP support assessing the sufficient conditions of taxonomy evaluation upon completion of Nickerson et al.’s (2013) taxonomy development method (i.e., that ends/restarts after Step 14) that depends on the taxonomy’s evaluation goal(s), purpose(s) and targeted user group(s). This implies adequately configuring an evaluation (Step 15) and subsequently performing it (Step 16). To observe and measure how well the taxonomy supports target user group(s) in achieving the intended purpose(s) (Step 17), researchers need to configure and perform an ex post evaluation in light of the evaluation goal(s). In DSR, evaluation is critical and should challenge the usefulness of the artefact (Hevner et al. 2004; Prat et al. 2015; Venable et al. 2016), here the usefulness of taxonomies. In addition, the evaluation in DSR depends on the type of artefact which is reflected in the evaluation methods (Cleven et al. 2009; Hevner et al. 2004; Prat et al. 2015) and evaluation criteria (March and Smith 1995; Prat et al. 2015; Sonnenberg and vom Brocke 2012) for a given artefact type. As we consider taxonomies as models, from the DSR perspective, the evaluation of a taxonomy is the evaluation of a model. Furthermore, taxonomies are usually reported together with definitions or descriptions of the taxonomies’ dimensions and characteristics (i.e., constructs of the model). Consequently, the evaluation of taxonomies also involves the evaluation of the constructs. The configuration and performance of taxonomy evaluation require taxonomy-specific answers to the ‘why’, ‘how’, and ‘what’ of DSR evaluation (Prat et al. 2015; Venable et al. 2016). In Table 6 we provide an integrated overview of the steps from demonstration to evaluation along these three guiding questions. For more operational guidance, we describe in Appendix 3 commonly used, taxonomy-related evaluation methods and evaluation criteria. In this way, the ETDP provides an evaluation trajectory starting from ex ante evaluation by checking the necessary (Is it a taxonomy?) and sufficient condition (Is it an applicable taxonomy?) to an ex post evaluation (Is it a useful taxonomy?).
In summary, taxonomy evaluation involves configuring a triad of evaluation methods, evaluation criteria, and target values/thresholds in light of the taxonomy’s purpose(s). It further includes performing the evaluation with partners relevant to or recruited from the target user group(s) (e.g., Sun and Kantor 2006). This triad allows investigating whether the evaluation goal(s) have been met and whether the actual use of the taxonomy enables the structuring of a phenomenon of interest (i.e., the taxonomy is useful). Bearing the iterative nature of taxonomy design in mind, it is important to note that we have complemented the additional guidance for the ex post evaluation of taxonomies with another exit point (after Step 17) for (re-entering) and reiterating into the taxonomy building before Step 2, 4, 6 or 10. For example, when the ex post evaluation shows that the taxonomy is not useful and does not meet the evaluation goal(s).
TDR 20 to 23 help configuring and performing an evaluation that fits the taxonomy’s target user group(s), purpose(s) and evaluation goal(s).
5.6 Communication
Step 18 of the ETDP supports the reporting of both the process of designing a taxonomy and the resulting design product (i.e., the taxonomy). Furthermore, reporting a taxonomy involves providing visualizations that fit the purpose(s) and target user group(s) (Szopinski et al. 2020) as well as descriptions for each characteristic and dimension. Beside this taxonomy-specific communication, it is also important to consider communication that is specific to the phenomenon under consideration (e.g., Hevner et al. 2004). By doing so, researchers can ensure that their taxonomies serve as a structure-giving artefact for the relevant target user group(s) and towards understanding certain phenomena in domains of interest.
TDR 24 to 26 help communicating the taxonomy and making it accessible to the target user group(s), for example, for practitioners who wish to use the taxonomy for decision-making or for researchers who seek to build their research upon the taxonomy.
6 Discussion and Conclusion
Taxonomies are important structure-giving artefacts that support understanding, describing, and analyzing novel and existing phenomena. Given that taxonomies shape past, present, and future conceptualizations and given that they represent the most fundamental form of theories (Gregor 2006), they should be rigorously designed. However, eight years after the publication of Nickerson et al.’s (2013) widely applied taxonomy method and after analyzing 164 taxonomy articles in the IS context, we observed an inconsistent adoption of existing methods, an often non-transparent reporting of relevant design decisions, and a demand for ex post evaluations. Thus, we infer a need for advancing and extending existing methods and conclude that taxonomy designers would benefit from further guidance supporting a more rigorous design of taxonomies – a motivation that was shared by the taxonomy experts during the evaluative interviews. Against this backdrop, we provide prescriptive knowledge contributing to an update for taxonomy designers in the form of an ETDP and corresponding TDR. Building on these contributions, we highlight three major implications, namely (1) access of taxonomy design to relevant DSR knowledge and methods, (2) guidance for taxonomy evaluation, and (3) evolution of taxonomies across iterations.
First, the integration of the ETDP with Peffers et al. (2007) DSR methodology implies an enriched body of knowledge for taxonomy designers. While we are not the first to relate taxonomies to DSR (e.g., Snow and Reck 2016; Yang and Varshney 2017), our explicit integration of taxonomy design into DSR methodology strengthens the taxonomy designers’ access to and allows for better leveraging DSR knowledge for taxonomy design. This includes, among others, general guidance and good practices to anchor artefacts in the problem and solution space (e.g., vom Brocke et al. 2020), fundamental principles for design-oriented science (Hevner et al. 2004), and reasoning mechanisms to generalize knowledge based on a DSR project (Kuechler and Vaishnavi 2008). In addition to general, cross-phase guidance, there is also phase-specific guidance. For example, to define the problem (e.g., Maedche et al. 2019) as well as design (e.g., Gregory and Muntermann 2014), evaluate (e.g., Peffers et al. 2007; Prat et al. 2015), and communicate the artefact (e.g., Gregor and Hevner 2013).
Second, beyond accessible DSR guidance on evaluating artefacts in general, we provide artefact-specific guidance on evaluating taxonomies responding to Nickerson et al.'s (2013, p. 353) call to examine taxonomies “for [their] usefulness for the intended users and the intended purpose”. Thus, we not only advance, but also extend methodological guidance providing complementary support for the evaluation of taxonomies in the form of detailed demonstration and evaluation steps along the guiding questions ‘why’, ‘how’, and ‘what’ (Prat et al. 2015; Venable et al. 2016). Thereby, formal validity is demonstrated through the use of objective ending conditions. Applicability and, most importantly, usefulness are validated through the application of subjective ending conditions and evaluation goals. Hence, researchers are asked to evaluate a taxonomy’s usefulness as a design artefact (Hevner et al. 2004).
Rigorously building and evaluating artefacts opens avenues for deriving more general design knowledge (Baskerville and Pries Heje 2019). Likewise, such knowledge can be derived based on rigorously designed taxonomies. Taxonomies in the form of models can be classified as nascent design theories (level 2 contribution, Gregor and Hevner 2013). Such taxonomies describe a phenomenon of interest and provide preliminary knowledge on what can be done. In this way, they provide prescriptions for design and action (Gregor et al. 2020). For example, identifying and structuring software features using a taxonomy to prescribe how existing and new software should be (re)designed (see Schöbel et al. 2020 for an example). Taxonomies can also inform well-developed design theories (level 3 contribution, Gregor and Hevner 2013). A prerequisite for such theories is a sound foundation of the constructs and the relationships between these constructs. Taxonomies can assist with this conceptual grounding because the development of design theories is typically preceded by an identification, structuration, and definition of characteristics and dimensions (Goldkuhl 2004).
Third, our ETDP is characterized by an iterative nature and various exit and entry points for revising a taxonomy. Thus, the ETDP supports the advancement of taxonomies, including after an initial publication. This implies a more effective evolution of taxonomies building upon existing and creating new structure-giving knowledge to pass on to future taxonomy design projects (vom Brocke et al. 2020). This could involve researchers designing and publishing an initial version of a taxonomy (e.g., at a conference), with other researchers building upon their work to evaluate, refine, and eventually update that taxonomy. Such cumulative taxonomy design should always start by scanning existing taxonomies and by justifying why another taxonomy is needed. Taxonomies from other disciplines, that often build upon and consolidate one another, could serve as role models in terms of knowledge accumulation and evolution. Examples include biology and astronomy that focus on one major taxonomy of organisms and planets (McKelvey 1982, 1987). Looking ahead, the IS community might also want to further reflect on potential mechanisms to manage the growing landscape of taxonomies, for example, by consolidating standalone taxonomies covering the same phenomenon. In addition, our ETDP allows for early involvement of practitioners, for example, to identify real-world objects or to evaluate the taxonomy. Thereby, our work paves the way for ensuring practical relevance, as demanded by many IS scholars (e.g., Baskerville et al. 2018; Hevner et al. 2004; Moeini et al. 2019).
As with any research, our work is beset with limitations that stimulate future research. First, the ETDP allows for research-specific method configurations, for example, following the ideas of Fazal-Baqaie and Engels (2016). As we do not provide any guidance on when and how to configure the ETDP, future research should examine potential avenues for method configurations. Second, we derived and evaluated our TDR based on the information described in the articles we analyzed as well as from author knowledge and expert interviews. Future research should strive for evaluating the ETDP and the TDR in naturalistic settings, for example, following-up whether and to what extent our work is applied and perceived useful by taxonomy designer. Thus, a study similar to our status quo analysis should be conducted in a couple of years after publication. Third, in the systematic analysis of taxonomy articles, we may have missed relevant articles since we did not cover all potential keywords and sources, but focused only on the most relevant IS journals and conferences as well as on articles citing the original taxonomy development method. However, for validating the coverage of our results and for ensuring alignment with the existing body of knowledge, we performed a cross check with related methods and recommendations on taxonomy design from IS and other disciplines. This check indicates that the ETDP and TDR address additional aspects and needs that have been brought up in related guidance.
As understanding the multitude of new and fast-evolving phenomena is crucial for society and business alike, taxonomies will continue to gain importance. By advancing taxonomy design in IS, we explicitly value taxonomies as a relevant foundation for understanding and further theoretical developments towards theories for explaining or design and action (Bapna et al. 2004; Gregor 2006; Iivari 2007; McKelvey 1987; Posey et al. 2013). We are confident that our methodological update will assist fellow researchers in more rigorously designing taxonomies.
Notes
Please note that we subsume the process of taxonomy building and taxonomy evaluation as taxonomy design for the remainder of this study. When referring to the method proposed by Nickerson et al. (2013) we keep the method’s original name (i.e., taxonomy development method).
We refer to the research process as being transparent if the taxonomy building process is reported sufficiently detailed in an article (i.e., complies with all seven attributes, see (Table 2). The attributes are based on both DSR literature in general and taxonomy literature in particular (see Appendix 2 for specific references).
Similar to the research process, we refer to the research product as being transparent if the taxonomy itself is reported sufficiently detailed in an article (i.e., complies with all four attributes, see (
Table 2). The attributes are based on taxonomy literature (see Appendix 2 for specific references).
References
Addas S, Pinsonneault A (2015) The many faces of information technology interruptions: a taxonomy and preliminary investigation of their performance effects. Inf Syst J 25(3):231–273
Agogo D, Hess TJ (2018) “How does tech make you feel?”: a review and examination of negative affective responses to technology use. Eur J Inf Syst 27(5):570–599
Al-Debei MM, Avison D (2010) Developing a unified framework of the business model concept. Eur J Inf Syst 19(3):359–376
Alrige M, Chatterjee S (2015) Toward a taxonomy of wearable technologies in healthcare. In: Proceedings of the 10th International Conference on Design Science Research in Information Systems and Technology (DESRIST 2015), Dublin
Bailey KD (1994) Typologies and taxonomies: an introduction to classification techniques. Sage, Thousand Oaks
Bapna R, Goes P, Gupta A, Jin Y (2004) User heterogeneity and its impact on electronic auction market design: an empirical exploration. MIS Q 28(1):21–43
Barn R, Barn B (2016) An ontological representation of a taxonomy for cybercrime. In: Proceedings of the 24th European Conference on Information Systems, Istanbul
Baskerville R, Baiyere A, Gregor S, Hevner A, Rossi M (2018) Design science research contributions: finding a balance between artifact and theory. J Assoc Inf Syst 19(5):358–376
Baskerville RL, Kaul M, Storey VC (2015) Genres of inquiry in design-science research. MIS Q 39(3):541–564
Baskerville RL, Pries-Heje J (2019) Projectability in design science research. J Inf Technol Theor Appl 20(1):53–76
Becker J, Niehaves B (2007) Epistemological perspectives on IS research: a framework for analysing and systematizing epistemological assumptions. Inf Syst J 17(2):197–214
Bayona-Oré S, Calvo-Manzano JA, Cuevas G, San-Feliu T (2014) Critical success factors taxonomy for software process deployment. Softw Qual J 22(1):21–48
Beinke JH, Nguyen D, Teuteberg F (2018) Towards a business model taxonomy of startups in the finance sector using blockchain. In: Proceedings of the 39th International Conference on Information Systems, San Francisco
Brennan RL, Prediger DJ (1981) Coefficient kappa: some uses, misuses, and alternatives. Educ Psychol Meas 41(3):687–699
Chasin F, von Hoffen M, Cramer M, Matzner M (2018) Peer-to-peer sharing and collaborative consumption platforms: a taxonomy and a reproducible analysis. Inf Syst E-Bus Manag 16(2):293–325
Cledou G, Estevez E, Soares Barbosa L (2018) A taxonomy for planning and designing smart mobility services. Gov Inf Q 35(1):61–76
Cleven A, Gubler P, Hüner KM (2009) Design alternatives for the evaluation of design science research artifacts. In: Proceedings of the 4th International Conference on Design Science Research in Information Systems and Technology, Philadelphia
Daniel S, Midha V, Bhattacherhjee A, Sing SP (2018) Sourcing knowledge in open source software projects: the impacts of internal and external social capital on project success. J Strateg Inf Syst 27(3):237–256
Degrossi LC, Porto de Albuquerque J, dos Santos RR, Zipf A (2018) A taxonomy of quality assessment methods for volunteered and crowdsourced geographic information. Trans GIS 22(2):542–560
De Langhe B, Fernbach P (2019) The dangers of categorical thinking. Harv Bus Rev 97(5):80–92
Dellermann D, Calma A, Lipusch N, Weber T, Weigel S, Ebel P (2019) The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems. In: Proceedings of the 52th Hawaii International Conference on System Sciences, Wailea
Domínguez E, Pérez B, Rubio AL, Zapata MA (2019) A taxonomy for key performance indicators management. Comput Stand Interfaces 64:24–40
Doty DH, Glick WH (1994) Typologies as a unique form of theory building: toward improved understanding and modeling. Acad Manag Rev 19(2):230–251
Dremel C, Stöckli E, Wulf J, Herrmann A (2018) Archetypes of data analytics providers in the big data era. In: Proceedings of the 24th Americas Conference on Information Systems, New Orleans
Eickhoff M, Muntermann J, Weinrich T (2017) What do FinTechs actually do? A taxonomy of FinTech business models. In: Proceedings of the 38th International Conference on Information Systems, Seoul
Eldredge N, Cracraft J (1980) Phylogenetic patterns and the evolutionary process: method and theory in comparative biology. Columbia University Press, New york
Fazal-Baqaie M, Engels G (2016) Software processes management by method engineering with MESP. In: Kuhrmann M, et al (eds) Managing software process evolution: traditional, agile and beyond – how to handle process change. Springer, Cham
Fellmann M, Robert S, Büttner S, Mucha H, Röcker C (2017) Towards a framework for assistance systems to support work processes in smart factories. In: Proceedings of the IFIP International Cross-Domain Conference for Machine Learning and Knowledge Extraction, Reggio di Calabria
Frank U (2006) Towards a pluralistic conception of research methods in information systems research, ICB-Research Report, 7. Universität Duisburg-Essen, Institut für Informatik und Wirtschaftsinformatik, Essen
Gao F, Thiebes S, Sunyaev A (2018) Rethinking the meaning of cloud computing for health care: a taxonomic perspective and future research directions. J Med Internet Res 20(7):1–27
Gimpel H, Rau D, Röglinger M (2018) Understanding FinTech start-ups: a taxonomy of consumer-oriented service offerings. Electron Mark 28(3):245–264
Goldkuhl G (2004) Design theories in information systems: a need for multi-grounding. J Inf Technol Theor Appl 6(2):59–72
Goldkuhl G (2016) Separation or unity? Behavioral science vs. design science. In: Proceedings of the AIS SIGPRAG Pre-ICIS Workshop of the 37th International Conference on Information Systems, Dublin
Gomber P, Koch JA, Siering M (2017) Digital finance and FinTech: current research and future research directions. J Bus Econ 87(5):537–580
Goo J, Kishore R, Rao HR (2000) A content-analytic longitudinal study of the drivers for information technology and systems outsourcing. In: Proceedings of the 21st International Conference on Information Systems, Brisbane
Gregor S (2006) The nature of theory in information systems. MIS Q 30(3):611–642
Gregor S, Hevner AR (2013) Positioning and presenting design science research for maximum impact. MIS Q 37(2):337–356
Gregor S, Chandra Kruse L, Seidel S (2020) Research perspectives: the anatomy of a design principle. J Assoc Inf Syst 21(6):1622–1652
Gregory RW, Muntermann J (2011) Theorizing in design science research: inductive versus deductive approaches. In: Proceedings of the 32nd International Conference on Information Systems, Shanghai
Gregory RW, Muntermann J (2014) Heuristic theorizing: proactively generating design theories. Inf Syst Res 25(3):639–653
Hambrick DC (1984) Taxonomic approaches to studying strategy: some conceptual and methodological issues. J Manag Inf Syst 10(1):27–41
Hanelt A, Hildebrandt B, Polier J (2015) Uncovering the role of IS in business model innovation: a taxonomy-driven approach to structure the field. In: Proceedings of the 23rd European Conference on Information Systems, Münster
Hempel CG (1966) Philosophy of natural science. Prentice-Hall, Upper Saddle River
Herterich MM, Buehnen T, Uebernickel F, Brenner W (2016) A taxonomy of industrial service systems enabled by digital product innovation. In: Proceedings of the 49th Hawaii International Conference on System Sciences, Koloa
Hevner AR, March ST, Park J, Ram S (2004) Design science in information systems research. MIS Q 28(1):75–105
Hevner AR, Chatterjee S (2010) Design research in information systems. Springer, Boston
Holler M, Uebernickel F, Brenner W (2017) Defining archetypes of e-collaboration for product development in the automotive industry. In: Proceedings of the 25th European Conference on Information Systems, Guimarães
Iivari J (2007) A paradigmatic analysis of information systems as a design science. Scand J Inf Syst 19(2):39–64
Janssen A, Passlick J, Cardona DR, Breitner MH (2020) Virtual assistance in any context. Bus Inf Syst Eng 62(3):211–225
Jöhnk J, Röglinger M, Thimmel M, Urbach N (2017) How to implement agile IT setups: a taxonomy of design options. In: Proceedings of the 25th European Conference on Information Systems, Guimarães
Kazan E, Tan C-W, Lim ET, Sørensen C, Damsgaard J (2018) Disentangling digital platform competition: the case of UK mobile payment platforms. J Manag Inf Syst 35(1):180–219
Keller R, König C (2014) A reference model to support risk identification in cloud networks. In: Proceedings of the 35th International Conference on Information Systems, Auckland
Krieger F, Drews P (2018) Leveraging big data and analytics for auditing: towards a taxonomy. In: Proceedings of the 39th International Conference on Information Systems, San Francisco
Kuechler B, Vaishnavi V (2008) On theory development in design science research: Anatomy of a research project. Eur J Inf Syst 17(5):489–504
Kunst K, Vatrapu R (2014) Towards a theory of socially shared consumption: literature review, taxonomy, and research agenda. In: Proceedings of the 22rd European Conference on Information Systems, Tel Aviv
Küpper T, Jung R, Lehmkuhl T, Wieneke A (2014) Features for social CRM technology: an organizational perspective. In: Proceedings of the 20th Americas Conference on Information Systems, Savannah
Kutzner K, Schoormann T, Knackstedt R (2018) Digital transformation in information systems research: a taxonomy-based approach to structure the field. In: Proceedings of the 26th European Conference on Information Systems, Portsmouth
Lakoff G (1987) Women, fire, and dangerous things: what categories reveal about the mind. University of Chicago, Chicago
Lakoff G, Johnson M (1999) Philosophy in the flesh: the embodied mind and its challenge to western thought. Basic Books, New York
Land L, Smith S, Pang V (2013) Building a taxonomy for cybercrimes. In: Proceedings of the 17th Pacific Asia Conference on Information Systems, Jeju Island
Land L, Smith S, Winchester D, Pang V (2014) The construction of identity offences taxonomy: an Australian context. In: Proceedings of the 25th Australasian Conference on Information Systems, Auckland
Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biom 33(1):159–174
Maedche A, Gregor S, Morana S, Feine J (2019) Conceptualization of the problem space in design science research. In: Proceedings of the 14th International Conference on Design Science Research in Information Systems and Technology, Worcester
March ST, Smith GF (1995) Design and natural science research on information technology. Decis Support Syst 15(4):251–266
Martin J (1983) Managing the data base environment. Prentice-Hall, Upper Saddle River
Mayer P (2017) A taxonomy of cross-language linking mechanisms in open source frameworks. Comput 99:701–724
McKay J, Marshall P, Hirschheim R (2012) The design construct in information systems design science. J Inf Technol 27(2):125–139
McKelvey B (1975) Guidelines for the empirical classification of organizations. Admin Sci Q 20(4):509–525
McKelvey B (1982) Organizational systematics: taxonomy, evolution, classification. University of California Press, Berkeley
McKelvey B (1987) Organizational systematics: taxonomic lessons from biology. Manag Sci 24(13):1428–1440
Moeini M, Rahrovani Y, Chan YE (2019) A review of the practical relevance of IS strategy scholarly research. J Strateg Inf Syst 28(2):196–217
Morana S, Pfeiffer J, Adam MTP (2020) User assistance for intelligent systems. Bus Inf Syst Eng 62(3):189–192
Mwilu OS, Prat N, Comyn-Wattiau I (2015) Taxonomy development for complex emerging technologies: the case of business intelligence and analytics on the cloud. In: Proceedings of the 19th Pacific Asia Conference on Information Systems, Singapore
Myers MD, Newman M (2007) The qualitative interview in IS research: examining the craft. Inf Org 17(1):2–26
Nakatsu RT, Grossman EB, Iacovou CL (2014) A taxonomy of crowdsourcing based on task complexity. J Inf Sci 40(6):823–834
Nickerson RC, Varshney U, Muntermann J (2013) A method for taxonomy development and its application in information systems. Eur J Inf Syst 22(3):336–359
Niehaves B (2007) On epistemological pluralism in design science. Scand J Inf Syst 19(2):93–104
Notheisen B, Willrich S, Diez M, Weinhardt C (2019) Requirement-driven taxonomy development: a classification of blockchain technologies for securities post-trading. In: Proceedings of the 52th Hawaii International Conference on System Sciences, Wailea
Oberländer AM, Röglinger M, Rosemann M, Kees A (2018) Conceptualizing business-to-thing interactions: a sociomaterial perspective on the internet of things. Eur J Inf Syst 27(4):486–502
Offermann P, Blom S, Schönherr M, Bub U (2010) Artifact types in information systems design science: a literature review. In: Proceedings of the 5th International Conference on Design Science Research in Information Systems and Technology, St. Gallen
Ojala H, Penttinen E, Collis J, Virtanen TH (2018) Design principles for standard business reporting (sbr)taxonomy development: evidence from Finland. Nord J Bus 67(1):4–26
Peffers K, Tuunanen T, Rothenberger MA, Chatterjee S (2007) A design science research methodology for information systems research. J Manag Inf Syst 24(3):45–77
Posey C, Raja U, Crossler RE, Burns AJ (2017) Taking stock of organisations’ protection of privacy: categorising and assessing threats to personally identifiable information in the USA. Eur J Inf Syst 26(7):585–604
Posey C, Roberts TL, Lowry PB, Bennett RJ, Courtney JF (2013) Insiders’ protection of organizational information assets: development of a systematics-based taxonomy and theory of diversity for protection-motivated behaviors. MIS Q 37(4):1189–1210
Prat N, Comyn-Wattiau I, Akoka J (2015) A taxonomy of evaluation methods for information systems artifacts. J Manag Inf Syst 32(3):229–267
Püschel L, Röglinger M, Schlott H (2016) What’s in a smart thing? Development of a multi-layer taxonomy. In: Proceedings of the 37th International Conference on Information Systems, Dublin
Raza U, Ahmad W, Khan A (2018) Transformation from manufacturing process taxonomy to repair process taxonomy: a phenetic approach. J Ind Eng Int 14(2):415–428
Recker J (2013) Scientific research in information systems. Springer, Heidelberg
Sangupamba OM, Prat N, Comyn-Wattiau I (2014) Business intelligence and big data in the cloud: opportunities for design-science researchers. In: Indulska M and Purao S (eds) Advances in Conceptual Modeling. ER 2014. Springer, Cham, pp 75–84
Sarkintudu SM, Ibrahim HH, Abdwahab AB (2018) Taxonomy development of blockchain platforms: information systems perspectives. In: Proceedings of the 3rd International Conference on Applied Science and Technology, Georgetown
Schäffer T, Stelzer D (2017) Towards a taxonomy for coordinating quality of master data in product information sharing. In: Proceedings of the 22nd MIT International Conference on Information Quality, Little Rock
Schoormann T, Behrens D, Knackstedt R (2017) Sustainability in business process models: a taxonomy-driven approach to synthesize knowledge and structure the field. In: Proceedings of the 38th International Conference on Information Systems, Seoul
Schöbel SM, Janson A, Söllner M (2020) Capturing the complexity of gamification elements: a holistic approach for analysing existing and deriving novel gamification designs. Eur J Inf Syst 29(6):641–668
Siering M, Clapham B, Engel O, Gomber P (2017) A taxonomy of financial market manipulations: establishing trust and market integrity in the financialized economy through automated fraud detection. J Inf Technol 32(3):251–269
Simon HA (1996) The sciences of the artificial. MIT Press, Cambridge
Snow NM, Reck JL (2016) Developing a government reporting taxonomy. J Inf Syst 30(2):49–81
Sokal RR, Sneath P (1963) Principles of numerical taxonomy. Freeman, San Francisco
Sonnenberg C, vom Brocke J (2012) Evaluations in the science of the artificial: reconsidering the build-evaluate pattern in design science research. In: Proceedings of the 7th International Conference on Design Science Research in Information Systems and Technology, Las Vegas
Stachowiak H (1973) Allgemeine Modelltheorie. Springer, Wien
Sun Y, Kantor PB (2006) Cross-evaluation: a new model for information system evaluation. J Am Soc Inf Sci Technol 57(5):614–628
Szopinski D, Schoormann T, John T, Knackstedt R, Kundisch D (2019a) Software tools for business model innovation: current state and future challenges. Electron Mark 30:469–494
Szopinski D, Schoormann T, Kundisch D (2019b) Because your taxonomy is worth it: towards a framework for taxonomy evaluation. In: Proceedings of the 27th European Conference on Information Systems, Stockholm-Uppsala
Szopinski D, Schoormann T, Kundisch D (2020) Visualize different: towards researching the fit between taxonomy visualizations and taxonomy tasks. In: Proceedings of the 15th International Conference on Wirtschaftsinformatik, Potsdam
Templier M, Paré G (2018) Transparency in literature reviews: an assessment of reporting practices across review types and genres in top IS journals. Eur J Inf Syst 27(5):503–550
Terrenghi N, Schwarz N, Legner C (2018) Towards design elements to represent business models for cyber physical systems. In: Proceedings of the 26th European Conference on Information Systems, Portsmouth
Thiebes S, Kleiber G, Sunyaev A (2017) Cancer genomics research in the cloud: a taxonomy of genome data sets. In: Proceedings of the 4th International Workshop on Genome Privacy and Security, Orlando
Tilly R, Posegga O, Fischbach K, Schoder D (2017) Towards a conceptualization of data and information quality in social information systems. Bus Inf Syst Eng 59(1):3–21
Tönnissen S, Teuteberg F (2018) Towards a taxonomy for smart contracts. In: Proceedings of the 26th European Conference on Information Systems, Portsmouth
Usman M, Britto R, Börstler J, Mendes E (2017) Taxonomies in software engineering: a systematic mapping study and a revised taxonomy development method. Inf Softw Technol 85:43–59
Van de Ven AH, Johnson PE (2006) Knowledge for theory and practice. The Acad Manag Rev 31(4):802–821
Varshney U (2014) Mobile health: four emerging themes of research. Decis Support Syst 66:20–35
Varshney U, Nickerson RC, Muntermann J (2015) Towards the development of a taxonomic theory. In: Proceedings of the 21st Americas Conference on Information Systems, Fajardo
Varshney U, Nickerson RC, Muntermann J (2017) Of taxonomies and taxonomic theories. In: Proceedings of the 23rd Americas Conference on Information Systems, Boston
Venable J, Pries-Heje J, Baskerville R (2016) FEDS: a framework for evaluation in design science research. Eur J Inf Syst 25(1):77–89
vom Brocke J, Maedche A (2019) The DSR grid: Six core dimensions for effectively planning and communicating design science research projects. Electron Mark 29(3):379–385
vom Brocke J, Winter R, Hevner A, Maedche A (2020) Accumulation and evolution of design knowledge knowledge in design science research: a journey through time and space. J Assoc Inf Syst 21(3):9–24
Weking J, Hein A, Böhm M, Krcmar H (2018a) A hierarchical taxonomy of business model patterns. Electron Mark 30:447–468
Weking J, Stöcker M, Kowalkiewicz M, Böhm M, Krcmar H (2018b) Archetypes for Industry 4.0 business model innovations. In: Proceedings of the 24th Americas Conference on Information Systems, New Orleans
Werder K, Wang H-Y (2016) Towards a software product industry classification. In: Proceedings of the 15th International Conference on Intelligent Software Methodologies, Tools and Techniques, Larnaca
Whetten DA (1989) What constitutes a theoretical contribution? Acad Manag Rev 14(4):490–495
Yang AT, Varshney U (2017) Categorizing mobile health project evaluation techniques. In: Proceedings of the 23rd Americas Conference on Information Systems, Boston
Zrenner J, Hassan AP, Otto B, Marx Gómez JC (2017) Data source taxonomy for supply network structure visibility. In: Proceedings of the Hamburg International Conference of Logistics, Hamburg
Acknowledgements
We thank participants of the Research Seminar of the Information Systems Department at University of Lausanne, the International Conference on Design Science Research in Information Systems (DESRIST) 2019, European Conference on Information Systems (ECIS) 2019, and Hawaii International Conference on System Sciences (HICSS) 2020 for their valuable feedback on parts of this work. This work was partially supported by the German Research Foundation (DFG) within the Collaborative Research Center “On-The-Fly Computing” (CRC 901, project number 160364472SFB901).
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Corresponding author
Additional information
Accepted after two revisions by Alexander Maedche.
The authors are listed alphabetically.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kundisch, D., Muntermann, J., Oberländer, A.M. et al. An Update for Taxonomy Designers. Bus Inf Syst Eng 64, 421–439 (2022). https://doi.org/10.1007/s12599-021-00723-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12599-021-00723-x