B Software
B.1 NCA with R
The main NCA software is a free package for the R platform called NCA (Dul & Buijs, 2026), which supports conducting a quantitative analysis of the data. The package is built on several other R packages:
gplots (Warnes et al., 2024),
quantreg (Koenker, 2024),
KernSmooth (Wand, 2023),
lpSolve (Berkelaar et al., 2023),
ggplot2 (Wickham et al., 2024),
doParallel (Corporation & Weston, 2022),
foreach (Analytics & Weston, 2022a),
iterators (Analytics & Weston, 2022b),
plotly (Sievert et al., 2024),
truncnorm (Mersmann et al., 2023),
RSQLite (Müller et al., 2026).
The first version of this software was released just before the online publication of NCA’s core paper (Dul, 2016b) on July 15, 2015. Since then, the software has been updated several times to correct bugs, integrate new NCA developments, and make improvements based on feedback from users. The following versions have been released through 17 June 2026:
NCA_1.0 (2015-07-02)
NCA_1.1 (2015-10-10)
NCA_2.0 (2016-05-18)
NCA_3.0 (2018-08-01)
NCA_3.0.1 (2018-08-21)
NCA_3.0.2 (2019-11-22)
NCA_3.0.3 (2020-06-11)
NCA_3.1.0 (2021-03-02)
NCA_3.1.1 (2021-05-03)
NCA_3.2.0 (2022-04-05)
NCA_3.2.1 (2022-09-15)
NCA_3.3.0 (2023-02-06)
NCA_3.3.1 (2023-02-10)
NCA_3.3.2 (2023-06-27)
NCA_3.3.3 (2023-09-05)
NCA_4.0.0 (2024-02-16)
NCA_4.0.1 (2024-02-23)
NCA_4.0.2 (2024-11-09)
NCA_4.0.3 (2025-10-14)
NCA_4.0.4 (2025-11-14)
NCA_4.0.5 (2025-12-21)
NCA_5.0.0 (2026-03-20)
NCA_5.0.1 (2026-04-28)
NCA_5.0.2 (2026-06-17)
When the first digit of the version number increases, major changes are implemented. Version 1 was the first limited version, version 2 was the first comprehensive version, version 3 introduced the statistical test for NCA, and version 4 introduced new functions. The second digit refers to larger changes and the third digit minor changes. Until 31 December 2025, the software was downloaded 69,983 times from the CRAN website (the Comprehensive R Archive Network).
During the publication of this book, version 5.0.0 of the software is released. This version is an extensive update with new main functions and NCA-specific helper functions (name starts with nca_ ...) and helper functions that are general helper functions (name starts with nca_util_ ...). Version 5.0.2 is used in this book.
To facilitate the analysis, some main functions of the NCA software are:
nca_analysis: conducts the core analysis (estimating ceiling line, effect size, \(p\)-value, etc.).
nca_output: displays the results.
nca_outlier: evaluates potential outliers.
nca_difference: conducts statistical difference test for comparing two effect sizes.
nca_power: conducts a power analysis.
nca_random: generates a random dataset representing population necessity.
B.2 Additional R-code
For some specialized applications, the book uses additional helper functions that are not (yet) integrated in the NCA package. For example, functions are available with templates for tables and figures that can be used to compare NCA results with results of other methods (Chapter 11). This includes NERT for combining NCA results with regression results (Table 11.4), NEST for combining NCA results with QCA results (Section 11.6.4), and BIPMA for giving practical advice based on combined NCA and SEM results (Section 11.4.1.7).
Additional R code can be downloaded via the book-page: https://jandul.github.io/NCA/.
B.3 Demonstration of NCA with R
This demonstration assumes little or no knowledge about R and RStudio and helps the user run the main NCA functions mentioned above.120
Example
The demonstration uses an example of necessary conditions for innovation in buyer-supplier co-operation (Van der Valk et al., 2016). The dataset of this early example of an NCA application is part of the NCA software.121 This demonstration follows the four stages of the NCA approach:
B.3.1 Stage 1: Formulate the formal necessity hypothesis
It is assumed that the hypotheses of the example were developed according to the requirements of a formal necessity hypothesis (Chapter 7). Only a limited explanation is provided here.
H1: Contractual detail (\(X_1\)) is necessary for innovation performance (\(Y\)); high-high direction.
H2: Goodwill trust (\(X_2\)) is necessary for innovation performance (\(Y\)); high-high direction.
H3: Competence trust (\(X_3\)) is necessary for innovation performance (\(Y\)); high-high direction.
• Focal unit = Buyer-supplier relationship.
• Concept \(X_1\) = Contractual detail: the degree of specificity and completeness in a contract.
• Concept \(X_2\) = Goodwill trust: the confidence that the supplier intends to fulfill its role and will act fairly.
• Concept \(X_3\) = Competence trust: the confidence in the supplier’s ability/capability to fulfill agreed obligations.
• Concept \(Y\) = Innovation: the extent to which the supplier continuously improves the maintenance process and increases asset utilization.
• Theoretical domain = buyer-supplier relations in manufacturing.
• Causal explanation H1 = Detailed contracts create a minimum set of formal agreements that make high innovation performance possible. Without these agreements, the work cannot be reliably coordinated, protected, and adapted. Other factors cannot compensate for the absence of detailed contracts.
• Causal explanation H2 = Goodwill trust creates the minimum relational confidence that makes high innovation performance possible. Without goodwill trust, partners will not share sensitive information, voice concerns early, or collaborate openly; instead they protect themselves, which reduces learning, slows joint problem solving, and blocks the experimentation needed for innovation. Other factors cannot compensate for the absence of goodwill trust.
• Causal explanation H3 = Competence trust creates the minimum capability confidence that makes high innovation performance possible. Without competence trust, coordination becomes inefficient because work must be checked, reworked, or duplicated; errors increase, response times slow, and adaptation suffers, making it impossible to execute complex, uncertain innovation tasks reliably. Other factors cannot compensate for the absence of competence trust.
B.3.2 Stage 2. Collect data
• Study design = observational study.
• Sampling = convenience sampling of n = 48 buyer-supplier relations.
• Measurement = questionnaires for buyer informants.
• Dataset = nca.example2 (part of the NCA software for R).
Install and load the NCA package and data
After R and RStudio are installed and RStudio is opened, the script window can be used to type and run instructions. The first instruction is to install (download) the NCA package with the install.packages function in R. Installing the NCA package must be done once. Afterwards, for each new session, the NCA package must be loaded (activated) using the library function. R runs the script line by line. Text after the #-symbol is ignored and may be used for deactivating a line or for comments. To limit the computation time for this demonstration, the test.rep argument of the nca_analysis function for the number of resamples for NCA’s statistical tests is reduced.
# Install and load the NCA package
# install.packages("NCA") # remove "#" in front if package is not installed
library(NCA) # load the package
# Limit computation time
test.rep <- 10000 # set lower for less computation time;
# for final analysis select 10000After loading the NCA package, the example data in the package are loaded and afterwards renamed for convenience. The number of rows (cases) and the first rows of the dataset are shown as well.
# Load the data
data("nca.example2") # load the data
data <- nca.example2 # rename the data for convenience
nrow(data) # number of rows (cases)## [1] 48head(data) # display the first rows of the data## Innovation Contractual detail Goodwill trust Competence trust
## 1 3.57 3.24 2.71 4.0
## 2 3.57 2.71 2.43 3.0
## 3 1.29 2.29 4.00 4.0
## 4 2.14 4.14 3.71 4.0
## 5 1.00 2.43 3.29 3.5
## 6 3.43 1.86 3.86 4.5The dataset has 48 cases. Each row represents a case. The four columns correspond to the four variables of the study, starting with outcome Innovation, and followed by the three conditions. The variables are measured on Likert scales with scale values ranging from 1 to 5, except for Contractual detail where the Likert scale values ranges from 1 to 7.
B.3.3 Stage 3. Analyze data
Model specification
• Selected ceiling line = CE-FDH, because data are discrete (Likert scales).
• Selected bounding box = empirical scope to avoid possible risk of overestimating the effect size.
• Selected effect size threshold = 0.10 (often used when no specific threshold is available).
• Selected \(p\)-value threshold = 0.05 (often used in the social sciences).
Visual inspection
First, three \(XY\)-plots are produced for a visual inspection of the data, one for each condition. Although \(XY\)-plots can be done with the NCA package (see below), here it is first done with standard R functions:
# Create XY-plots for visual inspection
# Using base R functions (not part of the NCA package)
plot(data$`Contractual detail`, data$Innovation,
xlab = "Contractual detail",
ylab = "Innovation")
plot(data$`Goodwill trust`, data$Innovation,
xlab = "Goodwill trust",
ylab = "Innovation")
plot(data$`Competence trust`, data$Innovation,
xlab = "Competence trust",
ylab = "Innovation")![$XY$-plots for visual inspection [Data from @van2016contracts].](demo1_xy.png)
![$XY$-plots for visual inspection [Data from @van2016contracts].](demo2_xy.png)
![$XY$-plots for visual inspection [Data from @van2016contracts].](demo3_xy.png)
Since the directions of the three hypotheses are high-high (presence/high \(X\) is necessary for presence/high \(Y\)), it is expected that in each \(XY\)-plot the upper-left corner is empty. The visual inspection confirms this.
Furthermore, it is observed that for each plot the borderlines in the upper-left corner between the area with cases and the area without cases are somewhat irregular. This suggests the use of the CE-FDH stepwise linear ceiling line that can follow the border more precisely than a linear ceiling line, although the other default ceiling line (CR-FDH) could have been selected as well.
Another observation is that no clear outliers are present, although the two points in the upper-right corner may be candidates as both ceiling and scope outliers. Note that in the plot of Competence trust only one point is visible in the upper-right corner, although here two points have the same \(XY\)-combination (duplicates). The default advice to not remove potential outliers unless these are errors is followed. However, uncertainty about outliers warrants an outlier check.
The plots also show that all possible values of the outcome (1–5) are represented in the dataset. This does not apply for the conditions where in particular low values of the conditions are missing. If these values are theoretically possible, this suggests that also a theoretical scope corresponding to the Likert scale could be selected, which warrants a robustness check of it (Sections 4.3 and 9.3.1).
Based on the visual inspection of the \(XY\)-plots, it is decided to first conduct the primary analysis with the selected CE-FDH ceiling line, to keep all cases, and to use the empirical scope. Afterwards robustness checks are conducted to evaluate the sensitivity of the choices for the main conclusion. Particularly, the effects of the use of CR-FDH ceiling line, and of a theoretical scope corresponding to the Likert scale values is evaluated. Also, a detailed outlier analysis is done. This possibly results in a different way of handling the outliers than is done in the original primary analysis (removing outliers), which may result in a different conclusion about the hypotheses.
Estimate effect size and \(p\)-value: necessity-in-kind
The main analysis consists of estimating the three effect sizes and corresponding \(p\)-values for making a decision about the necessity-in-kind for each hypothesis. The analysis is done with the function nca_analysis. The selected name of the analysis is model1.
# Conduct NCA
set.seed(123) # for reproducible results
model1 <- nca_analysis(data,
x = c('Contractual detail',
'Goodwill trust',
'Competence trust'),
y = 'Innovation',
ceilings = 'ce_fdh',
corner = 1, # default upper-left corner
scope = NULL, # default empirical scope
test.rep = test.rep) ## Setting up parallelization, this might take a few seconds... Do test for : ce_fdh - Contractual detailDone test for: ce_fdh - Contractual detail
## Do test for : ce_fdh - Goodwill trustDone test for: ce_fdh - Goodwill trust
## Do test for : ce_fdh - Competence trustDone test for: ce_fdh - Competence trustThe nca_analysis instruction first specifies the dataset, then the names of the conditions, followed by the name of the outcome. Instead of names, also column numbers can be selected: x = c(2,3,4); y = 1. The ceilings argument selects the ceiling line. Given the high-high hypotheses, the expected corner for all conditions is the upper-left corner (corner = 1). The test.rep argument activates the \(p\)-value estimation by selecting the number of permutations to create the null distribution with which the observed effect size is compared. The selection of the number of permutations is a trade-off between \(p\)-value accuracy and computation time. For the final analysis 10,000 permutations are recommended.
After running nca_analysis, the progress of the computation is printed. When the computations are done, no output is shown yet.
The main summary of the output can be displayed by calling print(model1) or just model1. This prints the effect sizes and \(p\)-value for each condition.
# Print main results
print(model1)##
## ---------------------------------------------------------------------------## Effect size(s):## ce_fdh p
## Contractual detail 0.24 0.008
## Goodwill trust 0.31 0.002
## Competence trust 0.32 0.002
## ---------------------------------------------------------------------------The results show support for the three hypotheses because the effect sizes and the \(p\)-values satisfy the threshold values that were selected.
By using the nca_output function, more output can be displayed. The \(XY\)-plots with the ceiling lines can be shown as follows.
# Display XY-plots with ceiling lines
nca_output(model1, summaries = FALSE)
Figure B.2: \(XY\)-plots with CE-FDH ceiling line.
Evaluate model fit
With the argument summaries = TRUE, further details of the results are printed for each condition separately.
# Print detailed output
nca_output(model1, plots = FALSE, summaries = TRUE)##
## ---------------------------------------------------------------------------## NCA Parameters : Contractual detail - Innovation## ---------------------------------------------------------------------------
##
## Number of observations 48
## Scope 15.40
## Xmin 1.86
## Xmax 5.71
## Ymin 1.00
## Ymax 5.00
##
## ce_fdh
## Ceiling zone 3.654
## Effect size 0.237
## # above 0
## Slope
## Intercept
## p-value 0.008
## p-accuracy 0.002
##
## Complexity 5
## Fit 100 %
## Ceiling accuracy 100 %
## Noise 0 % (0)
## Exceptions 0 % (0)
## Support 33.3% (16)
## Spread 0.49
## Sharpness 1.00
##
## Abs. ineff. 10.690
## Rel. ineff. 69.416
## Condition ineff. 22.078
## Outcome ineff. 60.750
##
##
## ---------------------------------------------------------------------------## NCA Parameters : Goodwill trust - Innovation## ---------------------------------------------------------------------------
##
## Number of observations 48
## Scope 10.28
## Xmin 2.43
## Xmax 5.00
## Ymin 1.00
## Ymax 5.00
##
## ce_fdh
## Ceiling zone 3.152
## Effect size 0.307
## # above 0
## Slope
## Intercept
## p-value 0.002
## p-accuracy 0.000
##
## Complexity 5
## Fit 100 %
## Ceiling accuracy 100 %
## Noise 0 % (0)
## Exceptions 0 % (0)
## Support 33.3% (16)
## Spread 0.53
## Sharpness 1.00
##
## Abs. ineff. 6.676
## Rel. ineff. 64.946
## Condition ineff. 1.946
## Outcome ineff. 64.250
##
##
## ---------------------------------------------------------------------------## NCA Parameters : Competence trust - Innovation## ---------------------------------------------------------------------------
##
## Number of observations 48
## Scope 8
## Xmin 3
## Xmax 5
## Ymin 1
## Ymax 5
##
## ce_fdh
## Ceiling zone 2.570
## Effect size 0.321
## # above 0
## Slope
## Intercept
## p-value 0.002
## p-accuracy 0.000
##
## Complexity 3
## Fit 100 %
## Ceiling accuracy 100 %
## Noise 0 % (0)
## Exceptions 0 % (0)
## Support 37.5% (18)
## Spread 0.28
## Sharpness 1.00
##
## Abs. ineff. 5.140
## Rel. ineff. 64.250
## Condition ineff. 0.000
## Outcome ineff. 64.250From the output, only model fit is discussed and model fit metrics are extracted by using the nca_extract function.
# Extract model fit metrics
xs = c('Contractual detail', 'Goodwill trust', 'Competence trust')
ceiling <- "ce_fdh"
metrics <- c("Complexity", "Fit", "Ceiling accuracy", "Noise", "Exceptions",
"Support", "Spread", "Sharpness")
tab1 <- sapply(xs, function(x) {
sapply(metrics, function(p) {
nca_extract(model1, x = x, ceiling = ceiling, param = p)
})
})
# Print model fit metrics
tab1_df <- data.frame(Metric = rownames(tab1), tab1,
row.names = NULL, check.names = FALSE)
tab1_df## Metric Contractual detail Goodwill trust Competence trust
## 1 Complexity 5.00 5.00 3.00
## 2 Fit 100.00 100.00 100.00
## 3 Ceiling accuracy 100.00 100.00 100.00
## 4 Noise 0.00 0.00 0.00
## 5 Exceptions 0.00 0.00 0.00
## 6 Support 33.30 33.30 37.50
## 7 Spread 0.49 0.53 0.28
## 8 Sharpness 1.00 1.00 1.00By definition, CE-FDH does not allow observations above the ceiling such that model fit in terms of Fit, Ceiling accuracy, Noise, Exceptions, and sharpness is ‘best’. However, this comes at the cost of model Complexity (and thus a greater risk of overfitting). The points on or just below the ceiling line show acceptable level of Support (> 5%, see Table 6.3), while the Spread metric indicates that these points are weakly dispersed along the ceiling line (compared to a suggested minimum value of 0.5).
Robustness checks
To evaluate if the support for the hypotheses is robust, three robustness checks are done. First, the sensitivity of the results for selecting another plausible ceiling line is evaluated. The CR-FDH ceiling line is an attractive alternative because it models the ceiling line as a linear ceiling line with low Complexity of 1. The analysis is repeated with this line.
# Perform robustness check with other ceiling line
set.seed(123) # for reproducible results
model2 <- nca_analysis(data,
x = c('Contractual detail',
'Goodwill trust',
'Competence trust'),
y = 'Innovation',
ceilings = "cr_fdh",
corner = 1, # default upper-left corner
scope = NULL, # default empirical scope
test.rep = test.rep)
print(model2)
nca_output(model2, summaries = FALSE)
# Extract model fit metrics
xs = c('Contractual detail', 'Goodwill trust', 'Competence trust')
ceiling <- "cr_fdh"
metrics <- c("Complexity", "Fit", "Ceiling accuracy", "Noise", "Exceptions",
"Support", "Spread", "Sharpness")
tab2 <- sapply(xs, function(x) {
sapply(metrics, function(p) {
nca_extract(model2, x = x, ceiling = ceiling, param = p)
})
})
# Print model fit metrics
tab2_df <- data.frame(Metric = rownames(tab2), tab2,
row.names = NULL, check.names = FALSE)
tab2_df## Setting up parallelization, this might take a few seconds... Do test for : cr_fdh - Contractual detailDone test for: cr_fdh - Contractual detail
## Do test for : cr_fdh - Goodwill trustDone test for: cr_fdh - Goodwill trust
## Do test for : cr_fdh - Competence trustDone test for: cr_fdh - Competence trust##
## ---------------------------------------------------------------------------## Effect size(s):## cr_fdh p
## Contractual detail 0.19 0.008
## Goodwill trust 0.26 0.003
## Competence trust 0.21 0.010
## ---------------------------------------------------------------------------## Metric Contractual detail Goodwill trust Competence trust
## 1 Complexity 1.00000 1.00000 1.00000
## 2 Fit 79.17335 83.22455 64.66219
## 3 Ceiling accuracy 95.83333 91.66667 93.75000
## 4 Noise 4.20000 4.20000 6.20000
## 5 Exceptions 0.00000 4.20000 0.00000
## 6 Support 22.90000 16.70000 16.70000
## 7 Spread 0.49000 0.56000 0.38000
## 8 Sharpness 0.82000 0.78000 0.57000
Figure B.3: \(XY\)-plots with CR-FDH ceiling line.
The results show that the effect sizes are reduced, but that they are still above the selected threshold level. Also the \(p\)-value remains below the selected threshold level, such that the conclusion about the hypothesis does not change with changing ceiling line from CE-FDH to CR-FDH.
The model fit measures confirm that the CR-FDH line is attractive regarding complexity. Since the CR-FDH line is a trend line through the peers (upper-left edges of CE-FDH line), cases exist above this line (ceiling accuracy < 100%). Although most of these cases remain close to the ceiling according to the Noise metric, for Goodwill trust two cases are relatively far from the ceiling (see Exceptions metric). These cases are potential outliers and were also identified by visual inspection. The Support and Spread metrics are comparable to those for CE-FDH.
The second robustness check tests the sensitivity of the results for selecting another scope. The analysis is repeated with a theoretical scope according to the minimum and maximum values of the Likert scales, rather than with the empirical scope based on observed minima and maxima.
# Robustness check with different scopes corresponding to Likert scales
set.seed(123)
model3 <- nca_analysis(data,
x = c('Contractual detail',
'Goodwill trust',
'Competence trust'),
y = 'Innovation',
ceilings = "ce_fdh",
corner = 1, # default upper-left corner
scope = list (c(1,7,1,5), c(1,5,1,5), c(1,5,1,5)),
test.rep = test.rep)
print(model3)
nca_output(model3, summaries = FALSE)
# Extract model fit metrics
xs = c('Contractual detail', 'Goodwill trust', 'Competence trust')
ceiling <- "ce_fdh"
metrics <- c("Complexity", "Fit", "Ceiling accuracy", "Noise", "Exceptions",
"Support", "Spread", "Sharpness")
tab3 <- sapply(xs, function(x) {
sapply(metrics, function(p) {
nca_extract(model3, x = x, ceiling = ceiling, param = p)
})
})
# Print model fit metrics
tab3_df <- data.frame(Metric = rownames(tab3), tab3,
row.names = NULL, check.names = FALSE)
tab3_df## Setting up parallelization, this might take a few seconds... Do test for : ce_fdh - Contractual detailDone test for: ce_fdh - Contractual detail
## Do test for : ce_fdh - Goodwill trustDone test for: ce_fdh - Goodwill trust
## Do test for : ce_fdh - Competence trustDone test for: ce_fdh - Competence trust##
## ---------------------------------------------------------------------------## Effect size(s):## ce_fdh p
## Contractual detail 0.30 0.008
## Goodwill trust 0.55 0.002
## Competence trust 0.66 0.002
## ---------------------------------------------------------------------------## Metric Contractual detail Goodwill trust Competence trust
## 1 Complexity 5.00 5.00 3.00
## 2 Fit 100.00 100.00 100.00
## 3 Ceiling accuracy 100.00 100.00 100.00
## 4 Noise 0.00 0.00 0.00
## 5 Exceptions 0.00 0.00 0.00
## 6 Support 33.30 33.30 37.50
## 7 Spread 0.49 0.53 0.28
## 8 Sharpness 1.00 1.00 1.00
Figure B.4: \(XY\)-plots with CE-FDH ceiling line and theoretical scope.
As expected, the empty spaces have increased such that the effect sizes are considerably larger than the original effect sizes and thus again above the selected threshold level. Also, the \(p\)-values remain below the selected threshold level, such that the conclusion about the hypothesis does not change with changing from empirical scope to theoretical scope.
Third, the sensitivity of the results for a different handling of outliers is evaluated. First, potential outliers are identified, then a decision is made to accept them as outlier or not, and finally to decide to include them in the analysis or not. If the results of this analysis are different than the original choice, the analysis is repeated with the new choice.
An outlier is defined as a case that has a large influence on the effect size when it is removed. The first outlier analysis is done by removing one potential outlier at a time. The nca_outlier function has similar arguments as the nca_analysis function. One additional argument is plotly, which can be selected to obtain an interactive plot to evaluate the cases.
# Single outlier analysis
nca_outliers(data, 2, 1, plotly = TRUE, ceiling = 'ce_fdh') # Contractual detail## outliers eff.or eff.nw dif.abs dif.rel ceiling scope
## 1 6 0.24 0.29 0.05 21.8 X
## 2 45 0.24 0.27 0.03 13.4 X
## 3 9 0.24 0.25 0.02 6.7 X
## 4 18 0.24 0.25 0.01 3.8 X
## 5 5 0.24 0.25 0.01 3.6 X
## 6 2 0.24 0.24 0.00 1.1 X
## 7 14 0.24 0.24 0.00 1.1 Xnca_outliers(data, 3, 1, plotly = TRUE, ceiling = 'ce_fdh') # Goodwill trust## outliers eff.or eff.nw dif.abs dif.rel ceiling scope
## 1 2 0.31 0.35 0.04 13.9 X
## 2 44 0.31 0.32 0.01 4.1 X
## 3 5 0.31 0.32 0.01 3.6 X
## 4 42 0.31 0.31 0.01 1.7 X
## 5 45 0.31 0.31 0.00 1.6 X
## 6 14 0.31 0.31 0.00 1.2 Xnca_outliers(data, 4, 1, plotly = TRUE, ceiling = 'ce_fdh') # Competence trust## outliers eff.or eff.nw dif.abs dif.rel ceiling scope
## 1 2 0.32 0.35 0.03 8.2 X
## 2 44 0.32 0.34 0.02 5.8 X
## 3 5 0.32 0.33 0.01 3.6 XThe printed results show that there are no cases that could be classified as outliers, because the change in effect size is relatively moderate when the case is removed. The maximum change is an effect size increase of 21.8% when case 6 (the upper-left case for Contractual detail) is removed.122
The outlier analysis continues with a removal of two conditions at the same time, because two case closely together or in the upper-right corner are a possible outlier pair.
# Double outliers analysis (two potential outliers)
nca_outliers(data, 2, 1, plotly = TRUE, k = 2, ceiling = 'ce_fdh') # Contractual detail## outliers eff.or eff.nw dif.abs dif.rel ceiling scope
## 1 45 - 8 0.24 0.09 -0.15 -62.6 X X
## 2 6 - 45 0.24 0.32 0.08 35.2 X
## 3 6 - 9 0.24 0.30 0.07 28.5 X
## 4 6 - 18 0.24 0.30 0.06 26.4 X X
## 5 6 - 5 0.24 0.30 0.06 26.2 X X
## 6 6 - 10 0.24 0.30 0.06 26.1 X
## 7 6 - 2 0.24 0.30 0.06 25.2 X
## 8 6 - 14 0.24 0.29 0.05 22.9 X
## 9 45 - 9 0.24 0.29 0.05 22.3 X
## 10 6 0.24 0.29 0.05 21.8 X
## 11 6 - 8 0.24 0.29 0.05 21.8 X
## 12 6 - 38 0.24 0.29 0.05 21.8 X
## 13 6 - 31 0.24 0.29 0.05 21.8 X
## 14 6 - 1 0.24 0.29 0.05 21.8 X
## 15 6 - 28 0.24 0.29 0.05 21.8 X
## 16 6 - 42 0.24 0.29 0.05 21.8 X
## 17 6 - 24 0.24 0.28 0.04 18.0 X X
## 18 45 - 18 0.24 0.28 0.04 17.7 X X
## 19 45 - 5 0.24 0.28 0.04 17.5 X X
## 20 45 - 2 0.24 0.27 0.03 14.5 X
## 21 45 - 14 0.24 0.27 0.03 14.5 X
## 22 45 0.24 0.27 0.03 13.4 X
## 23 45 - 24 0.24 0.27 0.03 13.4 X
## 24 45 - 38 0.24 0.27 0.03 13.4 X
## 25 45 - 10 0.24 0.27 0.03 13.4 X## # Not showing 59 possible outliersnca_outliers(data, 3, 1, plotly = TRUE, k = 2, ceiling = 'ce_fdh') # Goodwill trust## outliers eff.or eff.nw dif.abs dif.rel ceiling scope
## 1 45 - 8 0.31 0.12 -0.18 -59.9 X X
## 2 2 - 1 0.31 0.41 0.10 32.8 X
## 3 2 - 5 0.31 0.36 0.06 18.1 X X
## 4 2 - 44 0.31 0.36 0.06 18.0 X
## 5 2 - 42 0.31 0.35 0.05 15.6 X
## 6 2 - 45 0.31 0.35 0.05 15.5 X
## 7 2 - 14 0.31 0.35 0.05 15.2 X
## 8 2 0.31 0.35 0.04 13.9 X
## 9 2 - 8 0.31 0.35 0.04 13.9 X
## 10 2 - 9 0.31 0.35 0.04 13.9 X
## 11 2 - 36 0.31 0.35 0.04 13.9 X
## 12 2 - 28 0.31 0.35 0.04 13.9 X
## 13 2 - 41 0.31 0.33 0.03 9.1 X X
## 14 44 - 5 0.31 0.33 0.02 7.9 X X
## 15 44 - 42 0.31 0.33 0.02 7.6 X
## 16 44 - 14 0.31 0.33 0.02 6.6 X
## 17 44 - 45 0.31 0.32 0.02 5.7 X
## 18 5 - 42 0.31 0.32 0.02 5.4 X X
## 19 5 - 45 0.31 0.32 0.02 5.3 X X
## 20 5 - 14 0.31 0.32 0.02 4.9 X X
## 21 44 0.31 0.32 0.01 4.1 X
## 22 44 - 41 0.31 0.32 0.01 4.1 X
## 23 44 - 8 0.31 0.32 0.01 4.1 X
## 24 44 - 9 0.31 0.32 0.01 4.1 X
## 25 44 - 36 0.31 0.32 0.01 4.1 X## # Not showing 33 possible outliersnca_outliers(data, 4, 1, plotly = TRUE, k = 2, ceiling = 'ce_fdh') # Competence trust## outliers eff.or eff.nw dif.abs dif.rel ceiling scope
## 1 8 - 45 0.32 0.14 -0.19 -57.9 X X
## 2 2 - 44 0.32 0.37 0.04 14.0 X
## 3 2 - 40 0.32 0.37 0.04 13.8 X
## 4 2 - 5 0.32 0.36 0.04 12.1 X X
## 5 2 - 30 0.32 0.36 0.03 10.9 X
## 6 44 - 5 0.32 0.35 0.03 9.7 X X
## 7 2 0.32 0.35 0.03 8.2 X
## 8 2 - 8 0.32 0.35 0.03 8.2 X
## 9 2 - 45 0.32 0.35 0.03 8.2 X
## 10 2 - 14 0.32 0.35 0.03 8.2 X
## 11 2 - 16 0.32 0.35 0.03 8.2 X
## 12 2 - 9 0.32 0.35 0.03 8.2 X
## 13 44 0.32 0.34 0.02 5.8 X
## 14 44 - 8 0.32 0.34 0.02 5.8 X
## 15 44 - 45 0.32 0.34 0.02 5.8 X
## 16 44 - 40 0.32 0.34 0.02 5.8 X
## 17 44 - 30 0.32 0.34 0.02 5.8 X
## 18 44 - 14 0.32 0.34 0.02 5.8 X
## 19 44 - 16 0.32 0.34 0.02 5.8 X
## 20 44 - 9 0.32 0.34 0.02 5.8 X
## 21 5 0.32 0.33 0.01 3.6 X
## 22 5 - 8 0.32 0.33 0.01 3.6 X X
## 23 5 - 45 0.32 0.33 0.01 3.6 X X
## 24 5 - 40 0.32 0.33 0.01 3.6 X
## 25 5 - 30 0.32 0.33 0.01 3.6 X## # Not showing 3 possible outliersIt turns out that the pair 8-45 considerably reduces the effect size for all conditions (> 50%) when this pair is removed. This pair indeed consists of the two cases in the upper-right corner identified by visual inspection. Given the large influence that removing this pair has on the effect size, the robustness check consists of removing these cases from the dataset and repeating the analysis.
# Remove two potential outliers and perform NCA again
data1 <- data[-c(8, 45), ]
set.seed(123)
model4 <- nca_analysis(data1,
c('Contractual detail',
'Goodwill trust',
'Competence trust'),
'Innovation',
ceilings = 'ce_fdh',
corner = 1,
test.rep = test.rep)## Setting up parallelization, this might take a few seconds... Do test for : ce_fdh - Contractual detailDone test for: ce_fdh - Contractual detail
## Do test for : ce_fdh - Goodwill trustDone test for: ce_fdh - Goodwill trust
## Do test for : ce_fdh - Competence trustDone test for: ce_fdh - Competence trustprint(model4)##
## ---------------------------------------------------------------------------## Effect size(s):## ce_fdh p
## Contractual detail 0.09 0.111
## Goodwill trust 0.12 0.142
## Competence trust 0.14 0.050
## ---------------------------------------------------------------------------nca_output(model4, summaries = FALSE)The results show that after removal of the two outliers, the effect sizes decrease considerably. The effect size for Contractual detail is below the threshold level. Also all \(p\)-value are above the threshold level of 0.05. Consequently, without the outliers, none of the hypotheses is supported. This implies that the original results can be considered fragile and do not justify making strong conclusions.
Bottleneck analysis: necessity-in-degree
Despite the fragility of the original results, for this demonstration the results of model 1 are used for conducting a bottleneck analysis.
To show the bottleneck table, the nca_output function includes the argument bottlenecks = TRUE. To suppress further printed and plotted output, the argument summaries = FALSE and plots = FALSE are added.
# Default bottleneck analysis with percentage range for X and Y
nca_output(model1, bottlenecks = TRUE, summaries = FALSE, plots = FALSE)##
## ---------------------------------------------------------------------------## Bottleneck CE-FDH (cutoff = 0)## Y Innovation (percentage.range)## 1 Contractual detail (percentage.range)## 2 Goodwill trust (percentage.range)## 3 Competence trust (percentage.range)## ---------------------------------------------------------------------------
## Y 1 2 3
## 0 NN NN NN
## 10 NN NN NN
## 20 NN NN NN
## 30 NN NN NN
## 40 NN NN NN
## 50 NN NN NN
## 60 NN NN NN
## 70 48.1 49.8 50.0
## 80 77.9 98.1 100.0
## 90 77.9 98.1 100.0
## 100 77.9 98.1 100.0## The first column of the bottleneck table is a set of values of the outcome \(Y\), and the next columns show the corresponding required levels of the conditions. The table can be read row by row to find the levels of the conditions that are required for a given level of the outcome. NN means that the condition is not necessary for the corresponding level of the outcome. By default, bottleneck \(X\) and \(Y\)-values are expressed as percentages of the range. This means that 100 corresponds to the maximum value, 0 to the minimum value, 50% to the middle value, etc.
The output shows that up to a level of 60% Innovation performance, none of the conditions is necessary for Innovation performance. However, at 70% Innovation performance, each condition must have a level of about 50%. For a higher level of Innovation performance, a level of about 80% of Contractual detail and nearly maximum levels of Goodwill trust and Competence trust are required.
As discussed in Section 9.11, the values can also be expressed as actual values, percentiles, and percentage of maximum, which allows useful interpretations of the bottleneck table.
An alternative bottleneck table is one with levels of outcome and the conditions expressed as actual values. To obtain this table, the nca_analysis function must run with arguments that specify the desired type of value of \(X\) and \(Y\) for the bottleneck table (bottleneck.x and bottleneck.y). Furthermore, values of \(Y\) in the bottleneck table can be specified using the steps argument, for example, to specify the exact values of the anchors of the Likert scale of Innovation performance. To reduce computation time, the statistical analysis is not repeated (test.rep = FALSE or the argument is not included).
# Bottleneck analysis with actual values for X and Y
model5 <- nca_analysis(data,
c('Contractual detail',
'Goodwill trust',
'Competence trust'),
'Innovation',
ceilings = 'ce_fdh',
bottleneck.x = 'actual',
bottleneck.y = 'actual',
steps = seq(1, 5, by = 1)) # values of Likert scale
nca_output(model5, bottlenecks = TRUE, summaries = FALSE, plots = FALSE)##
## ---------------------------------------------------------------------------## Bottleneck CE-FDH (cutoff = 0)## Y Innovation (actual)## 1 Contractual detail (actual)## 2 Goodwill trust (actual)## 3 Competence trust (actual)## ---------------------------------------------------------------------------
## Y 1 2 3
## 1 NN NN NN
## 2 NN NN NN
## 3 NN NN NN
## 4 3.710 4.570 5.000
## 5 4.860 4.950 5.000## The output shows that Likert scale level 4 for Innovation performance is only possible if the Likert scale levels of Contractual detail, Goodwill trust, and Competence trust are at least 3.7, 4.5 and 5.0, respectively.
When actual values are used for the outcome and percentile values for the conditions, the formatting of the bottleneck table is somewhat different. The values of the conditions can now be interpreted as the percentage of cases that are unable to achieve the outcome. In brackets are the corresponding number of cases.
# Bottleneck analysis with actual values for Y and percentile values for X
model6 <- nca_analysis(data,
c('Contractual detail',
'Goodwill trust',
'Competence trust'),
'Innovation',
ceilings = 'ce_fdh',
bottleneck.x = 'percentile',
bottleneck.y = 'actual',
steps = seq(1, 5, by = 1))
nca_output(model6, bottlenecks = TRUE, summaries = FALSE, plots = FALSE)##
## ---------------------------------------------------------------------------## Bottleneck CE-FDH (cutoff = 0)## Y Innovation (actual)## 1 Contractual detail (percentile)## 2 Goodwill trust (percentile)## 3 Competence trust (percentile)## ---------------------------------------------------------------------------
## Y 1 2 3
## 1 NN (0) NN (0) NN (0)
## 2 NN (0) NN (0) NN (0)
## 3 NN (0) NN (0) NN (0)
## 4 58.3 (28) 83.3 (40) 79.2 (38)
## 5 83.3 (40) 93.8 (45) 79.2 (38)## This output shows that 58.3 percent of the cases (28 cases) did not achieve the required level of Contractual detail to make an Innovation performance of level 4 possible.
Advanced functions
Advanced functions in NCA for R that are demonstrated here are nca_difference, nca_power, and nca_random for statistical difference tests, power calculation, and random data generation, respectively.
Statistical difference tests
For certain studies it may be desirable to compare two effect sizes, for example, the difference in effect size between two time points, between two groups, or between two conditions. Assuming that the null hypothesis of no difference is not an unrealistic assumption, namely that both samples are drawn from the same population, NCA’s statistical difference test could be helpful in evaluating if the observed difference is compatible with the null of no difference (Section 5.3). For comparability of two effect sizes, a common scope should be selected. Depending on the question one tries to answer, this can be the pooled empirical scope (based on observed minimum and maximum scale values), a theoretical scope (e.g., based on theoretical minimum and maximum scale values), or a min-max normalized scope (e.g., \([0,1] \times [0,1]\)) that makes the scales dimensionless and numerically equal.
NCA’s contrast test compares effect sizes of two comparable conditions (in one sample).
# Contrast test: effect size difference between two conditions
# Common scope: normalized (for making the scales dimensionless)
# Min-max normalize data
scale <- c (0,1) # unit box
min_max <- c(min(data$Innovation), max(data$Innovation),
min(data$`Contractual detail`), max(data$`Contractual detail`),
min(data$`Goodwill trust`), max(data$`Goodwill trust`),
min(data$`Competence trust`), max(data$`Competence trust`)
)
data_n <- nca_util_normalize(data = data, scale = scale, min_max = min_max)
# Conduct test
set.seed(123)
difference_contrast <- nca_difference(data1 = data_n,
x = c("Goodwill trust", "Competence trust"),
y = "Innovation",
ceilings = "ce_fdh",
common_scope = c(0,1,0,1),
test.rep = 1000,
test.type = "contrast")## Doing ce_fdh ..................................................print(difference_contrast)##
## ---------------------------------------------------------------------------## Effect size (difference):## ce_fdh p
## Goodwill trust 0.31
## Competence trust 0.32
## Difference 0.01 0.128
## ---------------------------------------------------------------------------During computation, a message (Doing …) is shown. The output shows a minor and nonsignificant difference between the two effect sizes.
NCA’s independence test compares effect sizes of two independent samples (e.g., two subgroups). To simulate this situation it is assumed that the first set of cases in the dataset may have a different characteristic than the second set of cases and it is tested if the effect sizes of the two groups are different.
# Independence test: effect size difference between two datasets
data1 <- data[1:24, ]
data2 <- data[25:48, ]
# Common scope: pooled scope or common theoretical scope
# Conduct test with pooled empirical scope
# Common scope:
min_x <- min(min(data1$`Contractual detail`),min(data2$`Contractual detail`))
max_x <- max(max(data1$`Contractual detail`),max(data2$`Contractual detail`))
min_y <- min(min(data1$`Innovation`),min(data2$`Innovation`))
max_y <- max(max(data1$`Innovation`),max(data2$`Innovation`))
common_scope <- c(min_x,max_x,min_y,max_y)
# Conduct test
set.seed(123)
difference_independent <- nca_difference(data1 = data1,
data2 = data2,
x = c("Contractual detail"),
y = "Innovation",
ceilings = "ce_fdh",
common_scope = common_scope,
test.rep = 1000,
test.type = "independent")## Doing ce_fdh ..................................................print(difference_independent)##
## ---------------------------------------------------------------------------## Effect size (difference):## ce_fdh p
## Contractual detail.1 0.27
## Contractual detail.2 0.37
## Difference 0.10 0.331
## ---------------------------------------------------------------------------The results show no statistically significant difference between the groups.
NCA’s paired test compares effect sizes of two measurements (e.g., two time points, same group). This situation is simulated assuming that the data were obtained a T1 and by constructing a new sample (same sample size) where these cases get new scores for \(X\) and \(Y\) for T2.
# Paired test: effect size differences between 2 measurements
data1 <- subset(nca.example2, select = c(1,2)) # select X and Y for T1
# Simulate changes of X and Y at T2
data2 <- data1
set.seed(123)
data2$`Contractual detail` <- pmin(5, pmax(1, data2$`Contractual detail`
+ sample(-1:1, nrow(data2), TRUE)
)
)
data2$`Innovation` <- pmax(1, ifelse(data2$`Innovation`
>= 4, data2$`Innovation`
- 1, data2$`Innovation`)
)
# Conduct test with pooled empirical scope
# Common scope:
min_x <- min(min(data1$`Contractual detail`),min(data2$`Contractual detail`))
max_x <- max(max(data1$`Contractual detail`),max(data2$`Contractual detail`))
min_y <- min(min(data1$`Innovation`),min(data2$`Innovation`))
max_y <- max(max(data1$`Innovation`),max(data2$`Innovation`))
common_scope <- c(min_x,max_x,min_y,max_y)
# Conduct test
set.seed(123)
difference_paired <- nca_difference(data1=data1,
data2=data2,
x = "Contractual detail",
y = "Innovation",
ceiling = "ce_fdh",
common_scope = common_scope,
test.rep = 1000,
test.type = "paired")## Doing ce_fdh ..................................................print(difference_paired)##
## ---------------------------------------------------------------------------## Effect size (difference):## ce_fdh p
## Contractual detail.1 0.34
## Contractual detail.2 0.33
## Difference -0.01 0.887
## ---------------------------------------------------------------------------The output shows that the effect size is slightly different, and that the difference is not statistically significant.
Power analysis
For a planned study, a power analysis may be useful to estimate the probability that NCA’s statistical test rejects the null when necessity is true. Assumptions must be made for the (planned) sample size, the expected effect size, the slope of the true linear ceiling line, and the true distribution of data under the ceiling. Power also depends on the selected threshold \(p\)-value and ceiling-line estimation technique. For example, consider a planned study with sample size of 50, and an expected effect size of 0.2, with the default values of slope (1), distribution (uniform), threshold \(p\)-value (0.05) and ceiling technique (CE-FDH).
# Power analysis
set.seed(123)
power <- nca_power(n = 50, effect = 0.2) # This may take some time
print(power)## [1] "\r \rDoing batch 1 of 1 ..................................\r \r"
## [2] " n ES slope ceiling distribution.x distribution.y power"
## [3] "1 50 0.2 1 ce_fdh uniform uniform 0.98"The results show that power is 0.98 in the defined situation. This means that with high probability (though not guaranteed) the defined necessity effect will be detected in a random sample of n = 50 that is drawn from the population.
Random sample
For simulation studies (e.g., Monte Carlo simulations) the nca_random function produces a random sample from a population in which the necessity is represented by a linear ceiling line with given intercept and slope, and where the data under the ceiling and within the scope (unit box) have a uniform or truncated normal distribution. For example, a random sample of size 100 from a population where a necessary condition has a ceiling with an intercept of 0.2 and a slope of 1 with the default values of distribution under the ceiling (uniform) and upper left empty corner (1), the first six cases of the sample are:
# Get random sample with necessity
set.seed(123)
random <- nca_random (n = 100, intercepts = 0.2, slopes = 1)
head(random)## X Y
## 1 0.7883051 0.2875775
## 2 0.8830174 0.4089769
## 3 0.8924190 0.5281055
## 4 0.4566147 0.5514350
## 5 0.5726334 0.6775706
## 6 0.8998250 0.1029247Save results
When selected, the results from summaries, plots, and bottlenecks can be saved in pdf format using the argument pdf = TRUE in the nca_output function. The results are saved in the working directory.
# Save main results
nca_output(model6, summaries = TRUE, plots = TRUE, bottlenecks = TRUE, pdf = TRUE)A plotly plot can be saved from the Viewer window of RStudio by selecting ‘Download plot as png’, which saves the plot in the download folder of the computer. Saving is also possible via the Export tab \(\rightarrow\) Save as Image.
Help
The NCA package for R includes a manual to support users. The following instructions open the help.
# Access help documentation for the NCA package and specific functions
help(package = "NCA")
help("nca_analysis")
help("nca_output")
help("nca_extract")
help("nca_outliers")
help("nca_difference")
help("nca_random")
help("nca_power")B.3.4 Stage 4. Report results
The results of a study that applies NCA can be reported according to guidelines and recommendations presented in the SCoRe checklist (Section 10.3). This includes presenting NCA-specific elements such as:
Details of the formal necessity hypothesis and how it was developed.
The selected necessity model (primary ceiling line, bounding box).
The evaluation criteria for (non)-rejection of the hypothesis (effect size and \(p\)-value thresholds, and possibly the target outcome).
The result for each necessity hypothesis (including effect size, \(p\)-value, \(XY\)-plot and model fit).
The bottleneck tables for non-rejected hypotheses.
The details and results of the robustness checks.
For a pre-registered study the first three elements could be part of the study description.
B.4 Other software packages with NCA
Based on the NCA R package, a Stata module is available for conducting NCA (Spinelli et al., 2023). NCA is also part of the SmartPLS software for conducting NCA in combination with PLS-SEM (Magno et al., 2023; SmartPLS Development Team, 2023). These software packages enable conducting a basic NCA with platforms other than R, but lack more advanced functions and recent developments.
B.4.1 Demonstration of nca with Stata
It is assumed that the analyst has access to the Stata software. For this demonstration the same dataset is used as in the demonstration of NCA with R (see Section ??). In Stata the name of the NCA module is nca.
Prepare the analysis
After opening Stata, the nca package can be installed by entering in the Command window. Installing the nca package must be done once. If nca is already installed and a new version is available, the command is .
After installing the package, a folder (working directory) can be defined for the NCA-project using the command: , where ‘path’ is the specific path to the folder.
The package is now ready for use. The command in the command window opens a new window with help for the different nca functions.
Load the data
Enter to load the built-in dataset of nca. If there is a message ‘file ncaexample.dta not found’ the file location can be found with this command: and the command can be run as follows: , where [path] is obtained from the output of .
The dataset is now activated. Information about the active dataset appears in the Variables and Properties windows.
Estimate effect size and the \(p\)-value
To conduct NCA enter , in the Command window. This command first shows the names of the two conditions, followed by the name of the outcome. The next argument indicates that the \(p\)-value is estimated and that 10,000 permutations are used for it.
Create output
After the computations are done, the textual output is displayed in the Results window and the graphical output (\(XY\)-plots) in a pop-up window called ‘Graph’.
Perform the bottleneck analysis
The command for the bottleneck analysis while suppressing the standard nca textual and graphical output is .
The bottleneck table is displayed in the Results window with default values for conditions and outcome in terms of percentage of the range (0% is the lowest value; 100% is the highest value). The values can also be expressed as, for example, actual values and percentiles. The command for actual values for the outcome and percentiles for the conditions is: .