10 Reporting

10.1 Summary of this chapter

Previous chapters explained how to develop a formal necessity hypothesis (Chapter 7), and how to subsequently test it using empirical data (Chapter 8) and NCA’s data analysis approach (Chapter 9). This chapter describes how to report an NCA study. The goal is to report insights from the study, how these were obtained, and how they contribute to theoretical understanding and practical application. The chapter focuses on a written report for an academic audience (e.g., a journal article), but it can also be the basis for other reporting formats (e.g., presentations) and other audiences (e.g., practitioners, the general public). First, Section 10.2 distinguishes between several types of NCA publications (empirical, theoretical, methodological introductions, and practical) and provides examples of publications and recommendations. Second, Section 10.3 follows the common format of an academic publication (introduction – theory/hypotheses – methods – results – discussion) to present recommendations for conducting and reporting a high-quality NCA study. This section includes the SCoRe checklist for authors, editors, reviewers, and others to evaluate the quality of an NCA study and its reporting.

10.2 Types of NCA publications

NCA publications can be categorized into empirical, theoretical, methodological introductions, and practical publications. In the remainder of this chapter, it is assumed that the reader of the publication is a scholar or an engaged practitioner who is interested in NCA or in the substantive topic to which NCA is applied. While readers may not be experts on NCA, it is likely that they are familiar with the substantive topic.

10.2.1 Empirical publication

An empirical NCA study applies NCA’s methodology (e.g., necessity logic) and NCA’s method (e.g., for data analysis) to data that represent reality. The goal is to obtain a better understanding of a specific phenomenon of interest. In an academic setting, the intention is to contribute to the theoretical understanding of the phenomenon. In a practical setting, the intention is to contribute to solving a practical problem related to the phenomenon. A high-quality empirical NCA publication often has two main parts. The first part uses a necessity causal perspective for reframing current understanding of the phenomenon by distinguishing between essential (necessary) and contributing (substitutable) factors. The formulation of one or more formal necessity hypotheses is a central part of it. This implies that a formal hypothesis is formulated before the data analysis is conducted. While it is possible to conduct NCA in an inductive/exploratory manner (i.e., without a predefined hypothesis), this book focuses on the deductive approach for reasons mentioned in Chapter 7. The second part is the empirical testing of the hypotheses by providing evidence that supports or refutes the expected necessity relationship.

In an academic setting, the empirical publication is intended to provide a contribution to the academic knowledge and literature. The report of the study could contribute to a ‘conversation’ about a certain topic in the academic literature (e.g., Campbell & Aguilera, 2022; Lange & Pfarrer, 2017).97 The conversation starts with a common ground. This refers to an ongoing conversation about the problem/topic in the primary literature (i.e., what do we know?), and to which NCA contributes new insights. After that, an interesting complication is highlighted. This could, for example, be a missing element or an inconsistency in the academic conversation (i.e., what don’t we know?). Afterwards, it is explained why this complication is a concern (i.e., so what?), and it is reported how the course of action (i.e., NCA) addresses this complication. Finally, the contribution is described by showing how and why the new insights are now part of the ongoing conversation. For an empirical publication, the narrative to report the insights could focus on developing and testing a necessity hypothesis from scratch, or developing and testing an existing suggestion for a necessity relationship. Tables 10.1 and 10.2 provide examples of how such contributions could be formulated, respectively.


Table 10.1: Example of the formulation of the contribution of an empirical NCA publication that develops and tests a new necessity hypothesis.
Topic
Common ground (What do we know?)
Several contributing factors to the outcome of interest are identified in the primary literature that describes a complex multicausal phenomenon.
Complication (What don’t we know?)
We do not know which factors are essential.
Concern (So what?)
If the essential factor is not known, we do not know if it is present or not. If it is not present the outcome will not occur no matter the other factors. Acting on other factors than the bottleneck factor is a waste of resources.
Course of action
We analyze the phenomenon from the causal perspective of necessity by developing a formal necessity hypothesis using the NCA methodology and testing it with the NCA method.
Contribution
The phenomenon is now described in more detail by distinguishing between factors that contribute to the outcome and can be substituted (as commonly done), and essential factors that must be in place.


Table 10.2: Example of the formulation of the contribution of an empirical NCA publication that further develops and tests an existing necessity hypothesis.
Topic
Common ground (What do we know?)
A necessity relationship is suggested in the primary literature to describe a phenomenon.
Complication (What don’t we know?)
But necessity logic was not used when empirically testing that relationship, so we do not know if it is a true necessity relationship.
Concern (So what?)
Whether or not the relationship is a necessity relationship makes a difference for theoretical understanding and practical action. If necessary, the factor is essential in theory and practice; if not, the factor ‘only’ contributes and can be substituted.
Course of action
We develop the proposed necessity relationship toward a formal necessity hypothesis using the NCA methodology and test it with the NCA method to ensure theory-method fit.
Contribution
There is now empirical evidence to support or reject the claim of a necessity relationship.

In an empirical NCA publication, the following sections (or similar) are often present:

  • Introduction: introduction of necessity logic/theory and description of the contribution.

  • Theory/hypotheses: development of (one or more) formal necessity hypotheses using NCA’s methodology for necessity causal logic.

  • Methods: description of the data analysis approach using the NCA method.

  • Results: presentation of NCA’s results including the conclusion about (non-)support of the hypotheses.

  • Discussion: description of the potential importance of the (non-)support of the hypotheses for theory and practice.

Appendix C provides an overview of empirical journal articles until the end of 2025 that apply NCA. These publications are presented here to illustrate in which fields NCA has been applied. They do not serve as good examples of high-level applications of NCA. Detailed recommendations for conducting and reporting an empirical NCA study at a high level are presented in Section 10.3.

10.2.2 Theoretical publication

In some cases, the theoretical implications of using necessity theory according to the NCA methodology to describe a phenomenon may be complicated enough to warrant a separate publication. In a theoretical NCA publication, the focus can be on formulating necessity theory or defining concepts. The main contribution is the use of necessity logic. In such a publication, attention may be paid to NCA’s specific characteristics of this logic, such as conditional logic with a causal assumption, necessity-in-degree, or the typicality necessity perspective. Unlike in an empirical publication, a theoretical publication does not include an analysis of empirical data.

Few examples exist of theoretical publications with NCA (Table 10.3).

Table 10.3: Examples of theoretical journal articles with NCA.
Year Author(s) Journal/Book Main contribution
2020 Swab, Sherlock, Markin and Dibrell Family Business Review Clarify and advance understanding of the Socio-emotional Wealth (SEW) construct in family business research.
2022 Andrevski and Miller Academy of Management Review Show that forbearance is conceptually different from counteraction, word response, and involuntary nonresponse and is a deliberate, strategic choice.

10.2.3 Methodological introduction publication

The goal of a methodological introduction publication is to introduce NCA in a specific field by explaining its methodology and method.

Since the introduction of NCA in 2016, publications have introduced the approach, or part of it, in specific fields, often illustrated with examples from that field. Examples of these publications are shown in Table 10.4.

Table 10.4: Examples of publications with methodological introductions of NCA.
Year Author(s) Journal/Book Main contribution
2019 Tóth, Dul and Li Annals of Tourism Research Introduction of NCA in tourism
2020 Tynan, Credé and Harms Learning and Individual Differences Introduction of NCA in education
2020 Dul, Karwowski and Kaufman book chapter (Edward Elgar) Introduction of NCA in creativity
2021 Dul, Hauff and Tóth book chapter (Edward Elgar) Introduction of NCA in marketing
2021 Hauff, Guerci, Dul and van Rhee Human Resource Management Journal Introduction of NCA in HRM
2022 Richter and Hauff Journal of World Business Introduction of NCA in international business
2022 Greco, Guilera, Maldonado-Murciano, Gómez-Benito and Barrios International Journal of Environmental Research and Public Health Introduction of NCA in public health
2023 Linder, Moulick and Lechner Entrepreneurship Theory and Practice Introduction of NCA in entrepreneurship
2023 Bokrantz and Dul Journal of Supply Chain Management Introduction of NCA in supply chain management
2023 Lee, Dul and Tóth book chapter (Emerald) Introduction of NCA in tourism and hospitality
2023 Lee and Lu Journal of Hospitality & Tourism Research Introducing necessity theorizing in hospitality and tourism research
2026 Conde Journal of Marketing Analytics Introduction of NCA in sales
2026 Marchetti, Koster, Dul Clinical Psychology Review Introduction NCA in clinical psychology

As yet, only a small selection of fields has been covered by such publications, while NCA has been applied in many fields and could be applied in even more. This means there is room for more publications introducing NCA to a specific field. Introducing NCA in a new specific field has particular added value if it can be explained that:

  • Necessity logic is particularly useful for the field in general or for specific topics and challenges (e.g., a list of potential topics/challenges that can benefit from NCA).

  • Necessity thinking already (often implicitly) exists in the literature of the field (e.g., a list of necessity statements from the field).

  • NCA is different from conventional methodologies and methods (e.g., by explaining the logic and the steps of NCA using an example from the field).

  • NCA brings new insights to the field (e.g., an example of applying NCA to a topic from the field).

10.2.4 Practical publication

A final category of publications is relatively scarce. In the first 10 years of NCA, the focus was on the development of the methodology and the introduction in the academic world. Although academic publications often stress the practical relevance and potential of NCA, its uptake in practice has only just started. Early adopters, innovators and pioneers may have used NCA in their practical settings, but such activities usually do not lead to public publications. Yet, experiences from the use of NCA in practice may help to further develop the method and yield valuable insights for both theory and practice. Some possibilities of using NCA in practice are discussed in Chapter 12.

10.3 SCoRe checklist and recommendations

The publications listed in Appendix C are examples of adoptions of NCA during the first decade of NCA’s existence. Applying an innovative methodological approach is courageous, but it can also be risky. “When an emerging method’s standards of good practice are not yet broadly known, an early and fast adoption of the method without deep understanding of its details may result in incorrect interpretations that depart from the original ones and may result in invalid conclusions” (Dul, 2022, p. 1). The advancement of NCA and the availability of tools for applying the method appropriately can help move NCA into the next phase. In light of this, the SCoRe checklist and its recommendations are presented to support application of the method at the highest possible level.

The checklist and recommendations are meant to stimulate high-quality empirical NCA studies and their reporting. The SCoRe checklist is a tool for authors, peers, reviewers, editors, and readers in general to evaluate the quality of the execution and reporting of an NCA study. The acronym refers to the three themes covered in the checklist: Strengthening (theoretical rigor), Conducting (data and analysis quality), and Reporting (transparency). The checklist can help to identify study parts that are well developed, and those that need improvement.

The checklist is accompanied by best-practice guidelines. For each checklist item, recommendations are provided that can be used by analysts and authors for conducting their NCA study and for their publication about it. SCoRe can also help reviewers and others to provide constructive feedback.

The checklist and recommendations can be used for any empirical publication that uses NCA, whether NCA is the only method or is combined with other methods. In Sections 11.4.2 and 11.6.5, additional recommendations are given for applying NCA in combination with SEM and QCA, respectively.

SCoRe builds on several earlier works with basic guidelines for conducting and reporting NCA studies (Dul et al., 2023), recent developments in NCA reasoning (Dul, 2024a), necessity theorizing (Bokrantz & Dul, 2023), sampling for NCA (Dul, 2024b), the use of archival data in NCA (Dul et al., 2024), and the combined application of NCA and QCA (Dul, 2022) or PLS-SEM (Hauff et al., 2024; Richter, Hauff, Ringle, et al., 2023). Additionally, it incorporates topics newly introduced in this book, such as the formulation of a formal necessity hypothesis (Chapter 7), the use of robustness checks (Section 9.12) and updated recommendations for the combined application of NCA with SEM (Section 11.4) and NCA with QCA (Section 11.5). SCoRe is based on current knowledge, but as NCA and its application are continuously developing, the checklist may be updated in the future.

Following the common structure of an empirical publication, the checklist and its recommendations have six parts: Introduction/General, Theory/Hypotheses, Methods – data, Methods – data analysis, Results, Discussion. Each part discusses 5–10 items to be addressed for a high-level empirical NCA publication. The items have different priority levels. Some are essential (‘must-haves’) that must be addressed in any NCA publication. If any of the must-haves is missing, the quality of the publication should be seriously questioned. Others are important (‘should-haves’) to increase the quality of the publication, and yet other items are the ‘nice-to-haves’. When addressing all items, the publication meets the highest quality standard from the perspective of applying NCA, necessary for publishing in a high-ranking journal. Some items can be easily addressed, others need more reflection. For each item, requirements, recommendations, and sources for further reading are provided.

Below the checklist items are presented, followed by the recommendations.98

10.3.1 Introduction/General

Table 10.5 is the checklist for the Introduction/General part of applying and presenting NCA. The introduction should clearly define why NCA is used, describe it correctly, distinguish it from other causal methods, and highlight its role in the study.

Table 10.5: SCoRe checklist – Questions for Introduction/General.
Item Priority Question Check
1 Must-have Are the goals and contributions of applying NCA explicitly stated? ☐
2 Must-have When referring to necessity, are only words that correctly describe a necessity relationship used? ☐
3 Must-have Is it explained that NCA is an approach that combines a specific causal logic (methodology) and the related data analysis technique (method)? ☐
4 Must-have Is a proper comparison made between NCA’s causal perspective and conventional causal perspectives (e.g., probabilistic sufficiency, configurational sufficiency)? ☐
5 Must-have In multimethod studies, is NCA described and used as an approach with a specific perspective on causality and data analysis, and not merely as a robustness check or an add-on to other methods (and vice versa)? ☐
6 Nice-to-have If the necessity analysis is a prominent part of the publication, do the title and abstract reflect this necessity perspective/NCA? ☐

The following recommendations help to achieve the goals of the checklist items:

  1. State the primary and (possibly) secondary goals and contributions of the empirical NCA study. The study may primarily contribute to theory development by, for example, developing and testing new necessity hypotheses, developing an existing necessity claim further toward a formal necessity hypothesis and testing it with NCA, or replicating results of previously tested necessity hypotheses. Secondary goals could be methodological (e.g., introducing NCA to a new field/phenomenon), or practical (e.g., using results of NCA for practically meaningful recommendations/insights).
    See this book: Chapter 7, Section 10.2.
    Suggested articles: Bokrantz & Dul (2023); Dul (2024a); Hollenbeck & Wright (2017); Köhler & Cortina (2023); Bergh et al. (2022).

  2. When referring to necessity relationships, use appropriate terminology. In practice, this means using equivalents/derivatives of the statement that \(X\) is necessary for \(Y\). Avoid ambiguous statements (\(X\) causes \(Y\), \(X\) correlates with \(Y\), \(X\) is associated with \(Y\), \(X\) affects \(Y\)), or incorrect (sufficiency-based) statements such as \(X\) produces \(Y\), \(X\) drives \(Y\), \(X\) increases/decreases \(Y\), etc.
    See this book: Table 7.1.

  3. Refer to NCA as a methodological approach that combines the theoretical perspective of necessity causality (rather than common probabilistic or configurational causality) with a corresponding data analysis method, ensuring theory-method fit. Emphasize that NCA entails more than just a data analysis or statistical method.
    See this book: Chapters 1, 2, 5.
    Suggested article: Dul (2024a).

  4. When, in the context of an NCA study, a comparison is made between NCA’s causal perspective and the causal perspective inferred from conventional regression-based/statistical approaches, use the term probabilistic sufficiency for the latter, and not just sufficiency. The term sufficiency suggests determinism (as applied in QCA’s configurational sufficiency logic). This recommendation is often violated in studies that combine NCA with SEM. When NCA is compared with QCA’s necessity analysis, use the term necessity analysis of QCA for the latter (and not NCA) and explain the differences between the necessity analysis of QCA (only necessity-in-kind) and that of NCA (also necessity-in-degree).
    See this book: Sections 2.3, 2.4.
    Suggested articles: Dul (2024a); Dul (2016a); Vis & Dul (2018); Dul (2022).

  5. In multimethod studies, use and refer to NCA as an approach with a specific perspective on causality and data analysis, and do not incorrectly refer to it as a robustness check or add-on to other methods (and vice versa). This recommendation is often violated in studies that combine NCA with SEM.
    See this book: Sections 2.7, 11.4, 11.5.
    Suggested article: Dul (2024a).

  6. If important parts of the publication and/or its conclusions are based on necessity logic and NCA, ensure that this is reflected in the title and abstract by using words that reflect necessity logic or NCA.
    See this book: Table 7.1.

10.3.2 Theory/Hypotheses

Table 10.6 is the Theory/Hypotheses part of the checklist. This part is essential as it uses the NCA methodology to develop a formal hypothesis. A causal explanation of the necessity relationship is a central part of it. If it is missing, there is no theoretical justification for the hypothesis and the hypothesis should be rejected, regardless of whether the other criteria (effect size, \(p\)-value) are satisfied (Chapter 7).

Table 10.6: SCoRe checklist – Questions for Theory/Hypotheses.
Item Priority Question Check
7 Must-have Is the hypothesized necessity relationship explicitly formulated as: X is necessary for Y, or similar? ☐
8 Must-have Are the four elements of a necessity theory precisely defined: concepts X and Y, direction of the hypothesis, focal unit, theoretical domain? ☐
9 Must-have Is a causal explanation provided WHY X is necessary for Y using necessity causal logic to justify the necessity hypothesis by addressing three questions? Why does X enable Y? Why does the absence of X lead to the absence of Y? Why is the absence of X not compensable? ☐
10 Must-have Are temporal aspects considered including specification of the temporal order of X and Y if this is not obvious? ☐
11 Should-have Is the existing literature about the relationship between X and Y re-considered from a necessity causal perspective? ☐
12 Nice-to-have If applicable, is an nc-symbol added near the arrow in a diagram that represents a necessity relationship? ☐
13 Nice-to-have Is it specified if rare exceptions are allowed (typicality perspective)? ☐

  1. Formulate the necessity relationship explicitly as a necessary condition hypothesis: \(X\) is necessary for \(Y\). For a deductive study, this is done before data analysis; for an explorative study this is done after data analysis when indications for a necessity relationship are found.
    See this book: Section 7.5.

  2. To obtain a formal necessity hypothesis that is ready for testing, precisely define the four elements of the necessity theory. (1) Define the concepts \(X\) and \(Y\) according to their meaning in the hypothesis. (2) Specify the hypothesis and its direction, which indicates which corner of the \(XY\)-plot is expected to be empty, particularly if the expected corner is not the default upper-left corner. (3) Specify the focal unit of which \(X\) and \(Y\) of the hypothesis are characteristics. (4) Define the theoretical domain where it is claimed/expected that the hypothesis will hold, given its boundary conditions. Note that the theoretical domain is usually larger than a population from which cases are selected.
    See this book: Section 3.2, Chapter 7, in particular Section 7.5.

  3. Explain why \(X\) is necessary for \(Y\) using a causal explanation (narrative) by answering three questions: (1) Why does \(X\) enable \(Y\) (\(X\) is an enabler). The existing literature may be a source of information; (2) Why does the absence of \(X\) lead to the absence of \(Y\) (\(X\) is a constraint); (3) Why is the absence of \(X\) not compensable (no substitution for \(X\)). Making the narrative complete is a creative process, supported by literature and experiences of scholars and practitioners. See this book: Chapter 7, in particular Section 7.5.3.

  4. If this is not self-evident, include a plausible explanation of the temporal order (first \(X\) then \(Y\)) in the description of why the necessity causal relationship exists. If the data support that \(X\) is necessary for \(Y\), explain why the alternative reverse explanation, namely that \(Y\) is sufficient for \(X\) (or in terms of necessity: absence of \(Y\) is necessary for absence of \(X\)) that would also be consistent with the expected empty space, is not plausible. Also consider other temporal aspects of the hypothesis such as whether \(X\) is necessary for the onset or continuation of \(Y\), or whether the hypothesis holds in all time periods.
    See this book: Section 2.5, Chapter 7, in particular Section 7.5.3.

  5. Review existing literature (that commonly describes probabilistic relationships between your concepts X and Y), to find hints for necessity causality. Avoid just describing, summarizing, or referring to probabilistic findings/ideas. Where possible, include quotes in the literature that hint to necessity logic. Avoid using sufficiency/probabilistic causal logic for making a justification for necessity causal logic. Note that only describing, summarizing, or quoting prior work (often based on probabilistic sufficiency logic) does not constitute a valid theoretical justification of a necessity hypothesis.
    See this book: Section 7.4.1, Table 7.1.

  1. In a diagram representing an (expected) necessity relationship between two variables using an arrow, add a relevant symbol (+nc+, −nc+, −nc−, or +nc−) near the arrow to show the direction of necessity.
    See this book: Section 3.5.
  1. Explain whether the necessity theory allows for rare exceptions (i.e., a typicality perspective on necessity), or whether any exception leads to a rejection of the theory (i.e., a deterministic perspective on necessity). The typicality perspective is not a solution for poor measurement and should not be used in a probabilistic manner (e.g., ignoring a certain percentage of cases above the ceiling).
    See this book: Sections 2.4.3, 9.8, 4.5.4.
    Suggested article: Dul (2024a).

10.3.3 Methods – data

Table 10.7 is the Methods – data part of the checklist. Data is input to NCA, and good input is needed for good output. NCA accepts any type for scores of \(X\) and \(Y\) that are valid (representing the condition and outcome formulated in the hypothesis) reliable and meaningful. The latter refers to the possibility that values are interpretable for theory and practice.

Table 10.7: SCoRe checklist – Questions for Methods – data.
Item Priority Question Check
14 Must-have Is the study design (observational study, longitudinal study, case study, experimental study) adequate? ☐
15 Should-have For large-n quantitative studies: are adequate sampling approaches used and is a pre-study power analysis reported? ☐
16 Should-have For small-n qualitative studies: are adequate NCA-specific approaches used for purposive case selection? ☐
17 Must-have Are adequate measurement approaches used for getting scores of X and Y, and are these measures practically interpretable? ☐
18 Must-have For simulation studies: are simulated data sampled from bounded distributions of X and Y? ☐
19 Must-have Are only data transformations that align with NCA employed? ☐

  1. Possible study designs for NCA include the large-n observational study (both \(X\) and \(Y\) are observed), the longitudinal study (\(X\) and \(Y\) data from different time points), the small-n case study (\(X\) and \(Y\) are observed), and the experiment (\(X\) is manipulated and \(Y\) is observed). The general principles for good study design apply. For case selection and experiments, there is an NCA-specific approach.
    See this book: Section 8.2.

  2. For large-n quantitative studies the preferred sampling approach from the theoretical domain is random sampling with good coverage of the population of interest. The general principles for good sampling apply. Report if an NCA-specific pre-study power analysis was done to estimate the required sample size for finding necessity if it exists. Note that doing a post-hoc power analysis makes no sense for the present study, but could inform future studies.
    See this book: Sections 5.6, 8.3.1.
    Suggested article: Dul (2024b).

  3. For small-n qualitative NCA studies, aim to use purposive selection of cases from the theoretical domain, with cases being selected based on the presence of \(Y\) or absence of \(X\). Always clearly state how cases were selected. Reflect on, and report the potential limitations when cases were not selected purposively based on the absence of \(X\) or presence of \(Y\).
    See this book: Sections 8.2.3, 8.3.2.
    Suggested article: Dul (2024b).

  4. Ensure a proper measurement approach with valid, reliable, and meaningful data (scores of \(X\) and \(Y\)). Data are input to NCA, not part of NCA. General principles for good data (collection) apply to an NCA study. For interpretation of results, it is important that the scores have a practical meaning in terms of the levels of \(X\) and \(Y\). Data are stored in a dataset. The dataset may consist of archival data.
    See this book: Section 8.4.
    Suggested article: Dul et al. (2024).

  5. For data generated by simulation, ensure that the variables \(X\) and \(Y\) are bounded (have minimum and maximum values). Simulated data drawn from, for example, a normal distribution would not be eligible whereas data from a uniform or truncated normal distribution would be appropriate.
    See this book: Sections 4.3, 5.4.

  6. Explain whether and how the data were transformed, and whether the transformation aligns with NCA. Affine transformations (including linear transformation) produce valid results. NCA’s data analysis does not require data transformation and non-transformed data are often more meaningful and better interpretable (e.g., levels of a Likert scale) than transformed data. This particularly applies to necessity-in-degree and interpretations of specific levels of \(X\) being necessary for specific levels of \(Y\). Refrain from conducting non-linear data transformations (e.g., log-transformation, logistic transformation) unless the transformed data represent \(X\) and \(Y\) as defined in the necessity hypothesis (e.g., log-transformed GDP). This recommendation may be violated in studies that combine NCA with QCA when the logistic transformation (S-curve) is applied mechanistically for calibration purposes.
    See this book: Section 8.5.2, Appendix D.

10.3.4 Methods – data analysis

The main part of the NCA method is the data analysis which differs from common data analysis approaches. Table 10.8 is the Methods – data analysis part of the checklist. In multimethod studies where the NCA method is combined with other methods, the analyses are independent. The results are considered in combination only during interpretation and discussion. Therefore, the checklist applies to all types of empirical applications of the NCA method.

Table 10.8: SCoRe checklist – Questions for Methods – data analysis.
Item Priority Question Check
20 Must-have Is the selected ceiling line specified and justified? ☐
21 Should-have Is the bounding box (with empirical or theoretical scope) deliberately selected and specified? ☐
22 Should-have Are the hypothesis evaluation criteria (thresholds for effect size, p-value, and possibly target outcome) deliberately specified and justified? ☐
23 Should-have Are the results of a visual inspection of the XY-plot reported? ☐
24 Must-have Is it explained how potential outliers are analyzed and handled, and are removed cases reported? ☐
25 Should-have Are all model fit parameters evaluated? ☐
26 Should-have Are all NCA-related parameters, terms and concepts properly used and described? ☐
27 Nice-to-have Is the utilized software, including its version number specified? ☐
28 Nice-to-have Is a short general description of NCA’s data analysis approach provided to serve readers who are less familiar with NCA? ☐

  1. Specify and justify the selected ceiling line by considering theoretical expectations (border is linear or non-linear), type of data (discrete or continuous), observed border pattern (regular or irregular), balance between accuracy and generalizability (avoiding overfitting), and quality of the data (data with/without measurement error, simulation data).
    See this book: Sections 4.3.2, 9.3.2.

  2. Specify and justify the selected bounding box (with empirical or theoretical scope). Commonly, the empirical scope is selected to avoid over-estimation of the effect size. The theoretical scope can be considered when \(X\) and \(Y\) are measured on bounded scales (e.g., Likert scales, percentages).
    See this book: Sections 4.3, 9.3.1.

  3. Specify and justify the selected threshold values for necessity effect size \(d\) and \(p\)-value for identifying necessity-in-kind. This threshold depends on the specific context of the study. If no argument is available for a specific value, common benchmark values may be used (\(d\) = 0.10; \(p\) = 0.05 with 10,000 permutations). Optionally, select a target outcome \(Y\) to evaluate a specific outcome of necessity-in-degree.
    See this book: Sections 4.4.1, 5.3, 9.4.1, 9.4.2, 9.4.3.
    Suggested article: Dul (2016b).

  4. Conduct the analysis with the selected ceiling line and scope to obtain an \(XY\)-plot for visual inspection. Evaluate whether the expected corner is empty, if the selected ceiling line is appropriate, if potential outliers are present, and what the data pattern in the rest of the plot is.
    See this book: Section 9.5.

  5. Conduct an outlier analysis with the selected ceiling line and scope to identify potential ceiling and scope outliers. If outliers are removed, report in detail which cases are removed and why, and compare the results with and without removing the outliers (see robustness checks).
    See this book: Section 9.8.
    Suggested article: Dul (2016b).

  6. Conduct the analysis with the selected ceiling line and scope and evaluate the model fit parameters (complexity, fit, ceiling accuracy, noise, exceptions, support, spread, sharpness), and compare it with benchmark values.
    See this book: Sections 4.5, 6.5, 9.9.

  7. Describe the NCA-related parameters, terms, and concepts properly, including using the term ‘permutation test’ (not terms like bootstrapping, (Monte Carlo) simulation, robustness check, or \(t\)-test) for NCA’s statistical test, and using ‘multiple NCA’ (not multivariate NCA) for analyses with several necessity hypotheses.
    See this book: Appendix A.

  8. Specify which software and which version of the software was used to conduct NCA. The version number for the NCA package in R can be obtained with the command packageVersion("NCA").
    See this book: Appendix B.

  9. Include a brief description of NCA’s main logic and data analysis elements (ceiling line, ceiling zone, scope, effect size) so that the study is easier to understand for readers unfamiliar with NCA. Consider adding references for further reading.
    Suggested articles with examples of short descriptions: Dul (2024b); Dul (2025).

10.3.5 Results

Table 10.9 is the Results part of the checklist where the results of the data analysis are presented, and the (non-)support of the hypothesis is evaluated.

Table 10.9: SCoRe checklist – Questions for Results.
Item Priority Question Check
29 Must-have Are XY-tables or XY-plots of all tested/explored relationships shown? ☐
30 Should-have Is only the ceiling line that is used to draw conclusions about the necessity relationship shown in the XY-plot? ☐
31 Must-have For large-n quantitative study: are the effect size and its p-value properly reported? ☐
32 Must-have Is the conclusion about necessity-in-kind based on three criteria: theoretical justification, effect size, and p-value? ☐
33 Should-have Is the bottleneck table reported and necessity-in-degree evaluated? ☐
34 Nice-to-have If applicable: is the meaning of NN and NA in the bottleneck table explained? ☐
35 Should-have Are the type of values used in the bottleneck analysis justified? ☐
36 Must-have Are the results of the robustness checks summarized? ☐

  1. Include the \(XY\)-tables or \(XY\)-plots of all tested/explored relationships, preferably in the main text. It is essential that readers are able to inspect them visually.
    See this book: Section 9.5.

  2. In the \(XY\)-plot(s), include only the ceiling line that is used to draw conclusions about the necessity relationship. Note that the default output of the NCA software in R includes two ceiling lines, but that only one is usually selected for drawing conclusions about the hypothesis (\(XY\)-plots with other ceiling lines that were explored before the final selection was made, or that were evaluated during the robustness checks could be provided separately).

  3. Report the estimated effect size and its \(p\)-value. Preferably, report the effect size in two decimal places. This ensures that the result contains meaningful information about the precision, and that there is no unwarranted level of accuracy. Report the estimated \(p\)-value, preferably with three decimal places for the same reasons. Give exact \(p\)-values (or if applicable \(p\) < 0.001) and do not use inequalities or stars (i.e., not p < 0.05 or *).

  4. Conclude about necessity-in-kind using three criteria: (1) Theoretical justification (the formal hypothesis), (2) Effect size not below the selected threshold value, (3) \(p\)-value below the selected threshold value. Note that for support all criteria apply and that missing just one of them is enough for a rejection. This implies, for example, that without theoretical justification the claim for a necessity relationship is rejected.
    See this book: Section 9.10.

  5. Report the bottleneck table only for necessary conditions that were not rejected. This facilitates the evaluation of necessity-in-degree: which level of \(X\) is necessary for which level of \(Y\).
    See this book: Section 9.11.

  6. Explain the meaning of NN or NA in the bottleneck table if it appears.
    See this book: Section 9.11.5.

  7. Justify the type of values for \(X\) and \(Y\) that are displayed in the bottleneck table (e.g., percentage of range, actual values, percentiles). In particular, the actual values of \(Y\) and percentile values of \(X\) might be informative about the percentage or number of cases unable to achieve a target outcome.
    See this book: Section 9.11.

  8. Conduct robustness checks and summarize the results: is the main conclusion about necessity (support or rejection of the hypothesis) sensitive to choices by the analyst regarding ceiling line, scope, effect size threshold, \(p\)-value threshold, removing/keeping outliers, different choices for the target outcome? Conclude if the results are robust or fragile.
    See this book: Section 9.12.

10.3.6 Discussion

Table 10.10 is the Discussion part of the checklist where the results are interpreted and their consequences for theory and practice and for desired future studies are discussed.

Table 10.10: SCoRe checklist – Questions for Discussion.
Item Priority Question Check
37 Must-have Is it explained why all tested necessary conditions were (not) rejected? ☐
38 Must-have For a multimethod study: is it explained how the results of NCA and of other methods complement each other? ☐
39 Must-have Referring to the goals of the study (see introduction): is it discussed if the intended contribution is realized and what the key insights are? ☐
40 Should-have Are the results of the bottleneck analysis explained in terms of necessity-in-degree? ☐
41 Should-have Are limitations of applying NCA mentioned? ☐
42 Should-have Are potential future studies with NCA discussed? ☐

  1. Summarize the results of using NCA in terms of which necessary conditions were tested/explored, and which were rejected or not rejected. Assuming that (methodological) errors are unlikely, provide theoretical explanations for the results, referring to the necessity theory and hypotheses formulated earlier.
    See this book: Section 7.5.3.

  2. When NCA is used in a causal-pluralism study by combining it with another method (e.g., regression-based methods or QCA), report how the results of NCA and the other method complement each other using NERT, NEST and BIPMA when applicable.
    See this book: Chapter 11.

  3. For the primary and possibly secondary goals (see introduction), discuss if the intended contribution is realized and what key insights are gained.
    See this book: Section 10.2.1.

  4. If necessity-in-kind is accepted, provide an interpretation of necessity-in-degree using the results of the bottleneck analysis. For example, explain what level of \(X\) is necessary for a given target level of \(Y\).
    See this book: Section 9.11.

  5. Mention the study’s limitations. Discuss general limitations of NCA (e.g., exclusive focus on necessity; potential sensitivity to outliers), and specific limitations of applying NCA in the current study, and how this may affect the findings.

  6. Discuss potential future studies with NCA, for example conducting studies in other parts of the theoretical domain to assess the generalizability of the current findings, or applying NCA in related substantive areas that could benefit from the necessity causal perspective.
    Suggested article: Köhler & Cortina (2023).

References

Bergh, D. D., Boyd, B. K., Byron, K., Gove, S., & Ketchen Jr, D. J. (2022). What constitutes a methodological contribution? Journal of Management, 48(7), 1835–1848. https://doi.org/10.1177/01492063221088235
Bokrantz, J., & Dul, J. (2023). Building and testing necessity theories in supply chain management. Journal of Supply Chain Management, 59, 48–65. https://doi.org/10.1111/jscm.12287
Campbell, J. T., & Aguilera, R. V. (2022). Why I Rejected Your Paper: Common Pitfalls in Writing Theory Papers and How to Avoid Them. Academy of Management Review, 47(4), 521–527. https://doi.org/10.5465/amr.2022.0331
Dul, J. (2016a). Identifying single necessary conditions with NCA and fsQCA. Journal of Business Research, 69(4), 1516–1523. https://doi.org/10.1016/j.jbusres.2015.10.134
Dul, J. (2016b). Necessary Condition Analysis (NCA): Logic and methodology of “necessary but not sufficient” causality. Organizational Research Methods, 19(1), 10–52. https://doi.org/10.1177/1094428115584005
Dul, J. (2022). Problematic applications of Necessary Condition Analysis (NCA) in tourism and hospitality research. Tourism Management, 93, 104616. https://doi.org/10.1016/j.tourman.2022.104616
Dul, J. (2024a). A different causal perspective with Necessary Condition Analysis. Journal of Business Research, 177, 114618. https://doi.org/10.1016/j.jbusres.2024.114618
Dul, J. (2024b). How to sample in Necessary Condition Analysis (NCA). European Journal of International Management, 23(1), 1–12. https://doi.org/10.1504/EJIM.2024.138446
Dul, J. (2025). Identifying the perfect predictor of absence of disease: A shift toward necessary condition analysis in evidence-based medicine? In Journal of the American Academy of Child and Adolescent Psychiatry (pp. S0890–8567). https://doi.org/10.1016/j.jaac.2024.11.020
Dul, J., Hauff, S., & Bouncken, R. B. (2023). Necessary C"ondition Analysis (NCA): Review of research topics and guidelines for good practice. Review of Managerial Science, 17, 683–714. https://doi.org/10.1007/s11846-023-00628-x
Dul, J., Raaij, E. van, & Caputo, A. (2024). Advancing scientific inquiry through data reuse: Necessary Condition Analysis with archival data. Strategic Change, 33(1), 35–40. https://doi.org/10.1002/jsc.2562
Hauff, S., Richter, N. F., Sarstedt, M., & Ringle, C. M. (2024). Importance and performance in PLS-SEM and NCA: Introducing the combined importance-performance map analysis (cIPMA). Journal of Retailing and Consumer Services, 78, 103723. https://doi.org/10.1016/j.jretconser.2024.103723
Hollenbeck, J. R., & Wright, P. M. (2017). Harking, sharking, and tharking: Making the case for post hoc analysis of scientific data. Journal of Management, 43(1), 5–18. https://doi.org/10.1177/0149206316679487
Köhler, T., & Cortina, J. M. (2023). Constructive replication, reproducibility, and generalizability: Getting theory testing for JOMSR right. Journal of Management Scientific Reports, 1(2), 75–93. https://doi.org/10.1177/27550311231176016
Lange, D., & Pfarrer, M. D. (2017). Editors’ comments: Sense and structure—the core building blocks of an AMR article. Academy of Management Review, 42(3), 407–416. https://journals.aom.org/doi/10.5465/amr.2016.0225
Richter, N. F., Hauff, S., Ringle, C. M., Sarstedt, M., Kolev, A. E., & Schubring, S. (2023). How to apply necessary condition analysis in PLS-SEM. In Partial least squares path modeling: Basic concepts, methodological issues and applications (pp. 267–297). Springer. https://link.springer.com/chapter/10.1007/978-3-031-37772-3_10
Vis, B., & Dul, J. (2018). Analyzing relationships of necessity not just in kind but also in degree: Complementing fsQCA with NCA. Sociological Methods & Research, 47(4), 872–899. https://doi.org/10.1177/0049124115626179

  1. The ideas presented here are in line with typical approaches for presenting theoretical ideas using the 5 C’s of theory development: Constructs, Connections, Contingencies, Causal mechanisms, and Contributions (Lange & Pfarrer, 2017). However, this is only one way of structuring an academic argument about NCA. Academic writing is, after all, a matter of preference/style.↩︎

  2. An interactive version, and an AI-assisted version of the SCoRe checklist can be accessed via the book-page https://jandul.github.io/NCA/.↩︎