| Title: | Tidy Multiverse Analysis Made Simple |
|---|---|
| Description: | Extends 'multiverse' package (Sarma A., Kale A., Moon M., Taback N., Chevalier F., Hullman J., Kay M., 2021) <doi:10.31219/osf.io/yfbwm>, which allows users perform to create explorable multiverse analysis in R. This extension provides an additional level of abstraction to the 'multiverse' package with the aim of creating user friendly syntax to researchers, educators, and students in statistics. The 'mverse' syntax is designed to allow piping and takes hints from the 'tidyverse' grammar. The package allows users to define and inspect multiverse analysis using familiar syntax in R. |
| Authors: | Michael Jongho Moon [aut, cre], Haoda Li [aut], Mingwei Xu [aut], Nathan Taback [aut], Fanny Chevalier [aut], Alison Gibbs [ctb] |
| Maintainer: | Michael Jongho Moon <[email protected]> |
| License: | GPL (>= 3) |
| Version: | 0.2.3 |
| Built: | 2026-05-23 07:33:37 UTC |
| Source: | https://github.com/mverseanalysis/mverse |
mverse object.This method adds one or more branch conditions to
an existing mverse object. Branch conditions
are used to specify an option in one branch dependent
on an option in another branch.
add_branch_condition(.mverse, ...) ## S3 method for class 'mverse' add_branch_condition(.mverse, ...)add_branch_condition(.mverse, ...) ## S3 method for class 'mverse' add_branch_condition(.mverse, ...)
.mverse |
a |
... |
branch conditions. |
a mverse object.
Other branch condition functions:
branch_condition()
# Define branches and add them to an \code{mverse} object. y <- mutate_branch(alldeaths, log(alldeaths + 1)) distribution <- family_branch(poisson, gaussian) # You can match branching options by providing the options # the way provide them when defining branches. match_poisson <- branch_condition(alldeaths, poisson) mv <- mverse(hurricane) %>% add_mutate_branch(y) %>% add_family_branch(distribution) %>% add_branch_condition(match_poisson) summary(mv) # You can also condition to reject a pair of options by # setting reject = TRUE. match_log_lin <- branch_condition(log(alldeaths + 1), poisson, reject = TRUE) mv <- add_branch_condition(mv, match_log_lin) summary(mv)# Define branches and add them to an \code{mverse} object. y <- mutate_branch(alldeaths, log(alldeaths + 1)) distribution <- family_branch(poisson, gaussian) # You can match branching options by providing the options # the way provide them when defining branches. match_poisson <- branch_condition(alldeaths, poisson) mv <- mverse(hurricane) %>% add_mutate_branch(y) %>% add_family_branch(distribution) %>% add_branch_condition(match_poisson) summary(mv) # You can also condition to reject a pair of options by # setting reject = TRUE. match_log_lin <- branch_condition(log(alldeaths + 1), poisson, reject = TRUE) mv <- add_branch_condition(mv, match_log_lin) summary(mv)
mverse object.This method adds a family branch to an existing mverse object.
A family branch is used to define options for the analysis distributions
when using glm_mverse().
add_family_branch(.mverse, br)add_family_branch(.mverse, br)
.mverse |
a |
br |
a |
The resulting mverse object.
Other family branch functions:
family_branch
# Define a family branch. model_distributions <- family_branch( gaussian, poisson(link = "log") ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_family_branch(model_distributions)# Define a family branch. model_distributions <- family_branch( gaussian, poisson(link = "log") ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_family_branch(model_distributions)
mverse object.This method adds one or more filter branches to
an existing mverse object. Filter branches
are used to define options for conditions
for selecting subsets of data rows.
add_filter_branch(.mverse, ...)add_filter_branch(.mverse, ...)
.mverse |
a |
... |
|
The resulting mverse object.
Other filter branch functions:
filter_branch
# Define a filter branch. hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), !Name %in% c("Katrina"), !Name %in% c("Katrina"), TRUE # include all ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers)# Define a filter branch. hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), !Name %in% c("Katrina"), !Name %in% c("Katrina"), TRUE # include all ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers)
mverse object.This method adds a formula branch to an existing mverse object.
A formula branch is used to specify model structure options for the
analysis.
add_formula_branch(.mverse, br)add_formula_branch(.mverse, br)
.mverse |
a |
br |
a |
The resulting mverse object.
Other formula branch functions:
formula_branch
# Define a formula branch. model_specifications <- formula_branch( y ~ MasFem, y ~ MasFem + hurricane_strength, y ~ MasFem * hurricane_strength ) # Create a mverse, add the branch. mv <- create_multiverse(hurricane) %>% add_formula_branch(model_specifications)# Define a formula branch. model_specifications <- formula_branch( y ~ MasFem, y ~ MasFem + hurricane_strength, y ~ MasFem * hurricane_strength ) # Create a mverse, add the branch. mv <- create_multiverse(hurricane) %>% add_formula_branch(model_specifications)
mverse object.This method adds one or more mutate branches to
an existing mverse object. Mutate branches
are used to define options for adding a new
column to the analysis dataset.
add_mutate_branch(.mverse, ...)add_mutate_branch(.mverse, ...)
.mverse |
a |
... |
|
The resulting mverse object.
Other mutate branch functions:
mutate_branch
# Define mutate branches. hurricane_strength <- mutate_branch( # damage vs. wind speed vs.pressure NDAM, HighestWindSpeed, Minpressure_Updated_2014, # Standardized versions scale(NDAM), scale(HighestWindSpeed), -scale(Minpressure_Updated_2014), ) y <- mutate_branch( alldeaths, log(alldeaths + 1) ) # Create a mverse and add the branches. mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength) %>% add_mutate_branch(y) # You can also add multiple branches with a single call. mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength, y)# Define mutate branches. hurricane_strength <- mutate_branch( # damage vs. wind speed vs.pressure NDAM, HighestWindSpeed, Minpressure_Updated_2014, # Standardized versions scale(NDAM), scale(HighestWindSpeed), -scale(Minpressure_Updated_2014), ) y <- mutate_branch( alldeaths, log(alldeaths + 1) ) # Create a mverse and add the branches. mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength) %>% add_mutate_branch(y) # You can also add multiple branches with a single call. mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength, y)
Display the AIC and BIC score of glm regression
results across the multiverse.
## S3 method for class 'mverse' AIC(object, ..., k = 2) ## S3 method for class 'mverse' BIC(object, ...) AIC(object, ..., k = 2) BIC(object, ...)## S3 method for class 'mverse' AIC(object, ..., k = 2) ## S3 method for class 'mverse' BIC(object, ...) AIC(object, ..., k = 2) BIC(object, ...)
object |
a |
... |
ignored. for compatibility only. |
k |
ignored. for compatibility only. |
a multiverse table as a tibble
A branch condition conditions option x to
depend on option y. When the branch condition
is added to a mverse object, option x
is executed only when y is. Use reject = TRUE,
to negate the condition.
branch_condition(x, y, reject = FALSE)branch_condition(x, y, reject = FALSE)
x |
option 1 |
y |
option 2 |
reject |
if TRUE, the condition rejects universes with option 1 and option 2 |
A branch_condition object.
Other branch condition functions:
add_branch_condition()
# Example branches. y <- mutate_branch(alldeaths, log(alldeaths + 1)) model <- formula_branch(y ~ MasFem * strength, y ~ MasFem + strength) # Define a new branch condition. match_poisson <- branch_condition(alldeaths, poisson) # Define a branch condition that reject an option dependent on another. match_log_lin <- branch_condition(log(alldeaths + 1), poisson, reject = TRUE)# Example branches. y <- mutate_branch(alldeaths, log(alldeaths + 1)) model <- formula_branch(y ~ MasFem * strength, y ~ MasFem + strength) # Define a new branch condition. match_poisson <- branch_condition(alldeaths, poisson) # Define a branch condition that reject an option dependent on another. match_log_lin <- branch_condition(log(alldeaths + 1), poisson, reject = TRUE)
This method executes the analysis steps
defined in the mverse objected
across the entire multiverse.
execute_multiverse(.mverse, parallel = FALSE, progress = FALSE) ## S3 method for class 'mverse' execute_multiverse(.mverse, parallel = FALSE, progress = FALSE)execute_multiverse(.mverse, parallel = FALSE, progress = FALSE) ## S3 method for class 'mverse' execute_multiverse(.mverse, parallel = FALSE, progress = FALSE)
.mverse |
a |
parallel |
passed to |
progress |
passed to |
The resulting mverse object.
# Define a mutate branch. hurricane_strength <- mutate_branch( # damage vs. wind speed vs.pressure NDAM, HighestWindSpeed, Minpressure_Updated_2014, # Standardized versions scale(NDAM), scale(HighestWindSpeed), -scale(Minpressure_Updated_2014), ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength) # The branched variables are not populated across the multiverse yet. # Execute the multiverse; the variables are populated after the execution. execute_multiverse(mv)# Define a mutate branch. hurricane_strength <- mutate_branch( # damage vs. wind speed vs.pressure NDAM, HighestWindSpeed, Minpressure_Updated_2014, # Standardized versions scale(NDAM), scale(HighestWindSpeed), -scale(Minpressure_Updated_2014), ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength) # The branched variables are not populated across the multiverse yet. # Execute the multiverse; the variables are populated after the execution. execute_multiverse(mv)
extract returns a tibble of selected values
across the multiverse in a long format.
extract(...) ## S3 method for class 'mverse' extract( .mverse, columns = NULL, nuni = NULL, frow = NULL, include_branch_options = TRUE, ... )extract(...) ## S3 method for class 'mverse' extract( .mverse, columns = NULL, nuni = NULL, frow = NULL, include_branch_options = TRUE, ... )
... |
Ignored. |
.mverse |
a |
columns |
a character vector of column names to extract. |
nuni |
a positive integer for the number of universes to extract. |
frow |
proportion of rows to extract from each universe. |
include_branch_options |
when |
This method extracts data values across
the multiverse. You can specify a subset of data
to extract using columns, universe,
nuni, and frow.
You can specify the columns to extract from each
universe by passing the column names as a character
vector to columns. The default values is
NULL extracting all columns with branches.
Use universe to specify a set of universes
by their integer ids. Use nuni to specify
the number of universes to extract data from. The
method then selects the subset randomly. Specifying
universe manually will override nuni value.
By default, they are both set to NULL and
the method returns data from all universes.
Use frow to randomly extract a fraction of
data from each universe. The default value is NULL
and all rows are returned as they are. Note if you select
1 the method will return shuffle rows in each universe
before returning them. If frow is greater than 1,
the method randomly samples rows with replacement.
a tibble containing the selected columns across the multiverse.
# Define mutate branches. hurricane_strength <- mutate_branch( # damage vs. wind speed vs.pressure NDAM, HighestWindSpeed, Minpressure_Updated_2014, # Standardized versions scale(NDAM), scale(HighestWindSpeed), -scale(Minpressure_Updated_2014), ) y <- mutate_branch( alldeaths, log(alldeaths + 1) ) # Create a mverse and add the branches. mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength, y) execute_multiverse(mv) # Extract all branched columns from all universes extract(mv) # Specify the columns to extract from each universe using columns # You can select both branched and non-branched columns extract(mv, columns = c("hurricane_strength", "NDAM")) # Specify the universe to extract from using universe extract(mv, universe = 1) # Specify the number of universes to extract from using nuni # The universes are randomly selected extract(mv, nuni = 3) # Specify the proportion of data to extract from each universe using # frow. The rows are randomly selected extract(mv, frow = 0.7)# Define mutate branches. hurricane_strength <- mutate_branch( # damage vs. wind speed vs.pressure NDAM, HighestWindSpeed, Minpressure_Updated_2014, # Standardized versions scale(NDAM), scale(HighestWindSpeed), -scale(Minpressure_Updated_2014), ) y <- mutate_branch( alldeaths, log(alldeaths + 1) ) # Create a mverse and add the branches. mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength, y) execute_multiverse(mv) # Extract all branched columns from all universes extract(mv) # Specify the columns to extract from each universe using columns # You can select both branched and non-branched columns extract(mv, columns = c("hurricane_strength", "NDAM")) # Specify the universe to extract from using universe extract(mv, universe = 1) # Specify the number of universes to extract from using nuni # The universes are randomly selected extract(mv, nuni = 3) # Specify the proportion of data to extract from each universe using # frow. The rows are randomly selected extract(mv, frow = 0.7)
Create a new family branch.
family_branch(..., name = NULL)family_branch(..., name = NULL)
... |
branch definition expressions. |
name |
(optional) Name for the new family. |
a family_branch object.
Other family branch functions:
add_family_branch()
# Define a family branch. model_distributions <- family_branch( gaussian, poisson(link = "log") ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_family_branch(model_distributions)# Define a family branch. model_distributions <- family_branch( gaussian, poisson(link = "log") ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_family_branch(model_distributions)
Create a new filter branch.
filter_branch(..., name = NULL)filter_branch(..., name = NULL)
... |
branch definition expressions. |
name |
(optional) Name for the new filter. |
a filter_branch object.
Other filter branch functions:
add_filter_branch()
# Define a filter branch. hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), !Name %in% c("Katrina"), !Name %in% c("Katrina"), TRUE # include all ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers)# Define a filter branch. hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), !Name %in% c("Katrina"), !Name %in% c("Katrina"), TRUE # include all ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers)
The function specifies the model formula for fitting 'lm_mverse()' and
'glm_mverse()'. You can list the model specification formulae individually
or use covariates option paired with one or more formulae.
formula_branch(..., covariates = NULL, name = NULL)formula_branch(..., covariates = NULL, name = NULL)
... |
branch definition expressions. |
covariates |
(optional) A character vector of optional covariates. Each unique combination of the supplied covariates is translated into a unique branch option. See Details. |
name |
(optional) Name for the new formula. |
The optional argument covariates is allows you to specify a set of
optional covariates in addition to other independent variable such as
treatment variables and blocking variables which are specified using formula.
For each covariate provided, a branch is added to the multiverse with the
option to include or exclude the covariate in the model.
For example, formula_branch(y ~ x, covariates = c("c1", "c2")) creates
the following 4 model specifications:
y ~ x
y ~ x + c1
y ~ x + c2
y ~ x + c1 + c2
Here, y is the outcome variable and x may be a treatment
variable in an experiment setting. c1 and c2 may be additional
covariates about the experiment units that may or may not be relevant.
a formula_branch object.
Other formula branch functions:
add_formula_branch()
# Define a formula branch. model_specifications <- formula_branch( y ~ MasFem, y ~ MasFem + hurricane_strength, y ~ MasFem * hurricane_strength ) # Create a mverse, add the branch. mv <- create_multiverse(hurricane) %>% add_formula_branch(model_specifications) # Specify the covariates separately. model_specifications <- formula_branch( y ~ MasFem, covariates = c("hurricane_strength", "Year", "Category", "NDAM") ) model_specifications# Define a formula branch. model_specifications <- formula_branch( y ~ MasFem, y ~ MasFem + hurricane_strength, y ~ MasFem * hurricane_strength ) # Create a mverse, add the branch. mv <- create_multiverse(hurricane) %>% add_formula_branch(model_specifications) # Specify the covariates separately. model_specifications <- formula_branch( y ~ MasFem, covariates = c("hurricane_strength", "Year", "Category", "NDAM") ) model_specifications
glm_mverse fits glm across the multiverse
according to model specifications provided by formula_branch.
At least one formula_branch must have been added.
You can also specify the underlying error distribution and
the link function by adding a family_branch. If no
family_branch has been provided, it follows
the default behaviour of glm using the Gaussian
distribution with an identity link.
glm_mverse(.mverse, parallel = FALSE, progress = FALSE)glm_mverse(.mverse, parallel = FALSE, progress = FALSE)
.mverse |
a |
parallel |
passed to |
progress |
passed to |
A mverse object with glm fitted.
Other model fitting functions:
glm.nb_mverse(),
lm_mverse()
# Fitting \code{glm} models across a multiverse. hurricane_strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014 ) hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch( alldeaths ~ MasFem, alldeaths ~ MasFem + hurricane_strength ) model_distributions <- family_branch(poisson) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_mutate_branch(hurricane_strength) %>% add_formula_branch(model_specifications) %>% add_family_branch(model_distributions) %>% glm_mverse()# Fitting \code{glm} models across a multiverse. hurricane_strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014 ) hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch( alldeaths ~ MasFem, alldeaths ~ MasFem + hurricane_strength ) model_distributions <- family_branch(poisson) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_mutate_branch(hurricane_strength) %>% add_formula_branch(model_specifications) %>% add_family_branch(model_distributions) %>% glm_mverse()
glm.nb_mverse fits MASS::glm.nb across the multiverse
according to model specifications provided by formula_branch.
At least one formula_branch must have been added.
glm.nb_mverse(.mverse, parallel = FALSE, progress = FALSE)glm.nb_mverse(.mverse, parallel = FALSE, progress = FALSE)
.mverse |
a |
parallel |
passed to |
progress |
passed to |
A mverse object with glm.nb fitted.
Other model fitting functions:
glm_mverse(),
lm_mverse()
# Displaying the multiverse table with \code{glm.nb} models fitted. hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch(alldeaths ~ MasFem) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_formula_branch(model_specifications) %>% glm.nb_mverse() summary(mv)# Displaying the multiverse table with \code{glm.nb} models fitted. hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch(alldeaths ~ MasFem) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_formula_branch(model_specifications) %>% glm.nb_mverse() summary(mv)
A dataset for the study conducted by Jung et al. (2014) in Female hurricanes are deadlier than male hurricanes.
hurricanehurricane
A data frame with 94 rows and 12 variables:
Year in which the hurricane landed on U.S.
Name of the hurricane.
Femininity index of the hurricane name collected by Jung et al. (1 - very masculine; 11 - very feminine).
Minimum pressure of the hurricane at the time of landfall in the U.S. (original).
Minimum pressure of the hurricane at the time of landfall in the U.S. (updated).
Gender indicator for the hurricane name based
on MasFem index (1 - MasFem > 6; 0 otherwise).
Hurricane category on a scale of 1 to 5, with 5 being the most severe.
Number of fatalities.
Normalized damage in 2013 U.S. million dollars.
Time since hurricane.
Source from where the data was gathered.
Maximum wind speed.
Femininity index of the hurricane name collected by Simonsohn et al.
Normalized damage in 2015 U.S. million dollars.
The dataset was collected by Jung et al. in their study Female hurricanes are deadlier than male hurricanes. Their study didn't include hurricanes Katrina and Audrey which were deemed as outliers. Simonsohn et al. collected the extra data for Specification curve analysis including an additional femininity index based on an MTUrk survey and updated normalized damage amount in 2015 U.S. dollars.
This dataset includes data prepared by Jung et al. (2014) as well as those prepared by Simonsohn et al. (2020). Specifically, all data on Katrina and Audrey are from Simonsohn et al. (2020) except minimum pressure updated in 2014. They were retrieved from Continental United States Hurricane Impacts/Landfalls 1851-2021 table maintained by U.S. National Oceanic and Atmospheric Administration. Maximum wind speed, femininity index from MTUrk survey, and 2015 damage amounts are also from Simonsohn et al. (2020).
Kiju Jung, Sharon Shavitt, Madhu Viswanathan, and Joseph M. Hilbe. (2014). "Female hurricanes are deadlier than male hurricanes." Proceedings of the National Academy of Sciences, 111(24), 8782-8787. doi:10.1073/pnas.1402786111
Uri Simonsohn, Joseph P. Simmons, and Leif D. Nelson. (2020). “Specification curve analysis” Nature Human Behaviour, 4, 1208–14. doi:10.1038/s41562-020-0912-z
lm_mverse fits lm across the multiverse
according to model specifications provided by formula_branch.
At least one formula_branch must have been added.
lm_mverse(.mverse, parallel = FALSE, progress = FALSE)lm_mverse(.mverse, parallel = FALSE, progress = FALSE)
.mverse |
a |
parallel |
passed to |
progress |
passed to |
A mverse object with lm fitted.
Other model fitting functions:
glm.nb_mverse(),
glm_mverse()
# Fitting \code{lm} models fitted across a multiverse. hurricane_strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014 ) y <- mutate_branch( alldeaths, log(alldeaths + 1) ) hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch( y ~ MasFem, y ~ MasFem + hurricane_strength ) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_mutate_branch(hurricane_strength, y) %>% add_formula_branch(model_specifications) %>% lm_mverse()# Fitting \code{lm} models fitted across a multiverse. hurricane_strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014 ) y <- mutate_branch( alldeaths, log(alldeaths + 1) ) hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch( y ~ MasFem, y ~ MasFem + hurricane_strength ) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_mutate_branch(hurricane_strength, y) %>% add_formula_branch(model_specifications) %>% lm_mverse()
A multiverse tree diagram displays the branching combination
of all the branches added to the given mverse object
taking any branch conditions defined. The method also allows
zooming into a subset of branches using branches parameter.
multiverse_tree( .mverse, label = "none", branches = NULL, label_size = NULL, label_angle = 0, label_hjust = 0, label_vjust = 0 )multiverse_tree( .mverse, label = "none", branches = NULL, label_size = NULL, label_angle = 0, label_hjust = 0, label_vjust = 0 )
.mverse |
A |
label |
Display the branch option name when "name" or the definition when "code". No label is displayed when "none" (default). |
branches |
A character vector. Display a subset of branches when specified. Display all when NULL. |
label_size |
A numeric. Set size of option labels. |
label_angle |
A numeric. Rotate option labels. |
label_hjust |
A numeric. Set the horizontal justification of the node labels. |
label_vjust |
A numeric. Set the vertical justification of the node labels. |
A ggplot object displaying the multiverse tree.
{ # Display a multiverse tree with multiple branches. outliers <- filter_branch(!Name %in% c("Katrina", "Audrey"), TRUE) femininity <- mutate_branch(MasFem, Gender_MF) strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014, log(NDAM) ) y <- mutate_branch(alldeaths, log(alldeaths + 1)) model <- formula_branch(y ~ femininity * strength, y ~ femininity + strength) distribution <- family_branch(poisson, gaussian) mv <- mverse(hurricane) %>% add_filter_branch(outliers) %>% add_mutate_branch(femininity, strength, y) %>% add_formula_branch(model) %>% add_family_branch(distribution) multiverse_tree(mv) # Display a multiverse tree with branch conditions. match_poisson <- branch_condition(alldeaths, poisson) match_log_lin <- branch_condition(log(alldeaths + 1), gaussian) add_branch_condition(mv, match_poisson) add_branch_condition(mv, match_log_lin) multiverse_tree(mv) # You can adjust colour scale of the edges # using a ggraph::scale_edge_colour*() function. multiverse_tree(mv) + ggraph::scale_edge_colour_viridis( discrete = TRUE, labels = c("Distribution", "Model", "Strength", "Femininity", "Outliers", "y") ) # Display a multiverse tree for a subset of branches # with name label for each option. multiverse_tree(mv, branches = c("y", "distribution"), label = "name") # with code label for each option. multiverse_tree(mv, branches = c("y", "distribution"), label = "code") # adjusting size and orientation of the labels multiverse_tree(mv, branches = c("y", "distribution"), label = "name", label_size = 4, label_angle = 45) }{ # Display a multiverse tree with multiple branches. outliers <- filter_branch(!Name %in% c("Katrina", "Audrey"), TRUE) femininity <- mutate_branch(MasFem, Gender_MF) strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014, log(NDAM) ) y <- mutate_branch(alldeaths, log(alldeaths + 1)) model <- formula_branch(y ~ femininity * strength, y ~ femininity + strength) distribution <- family_branch(poisson, gaussian) mv <- mverse(hurricane) %>% add_filter_branch(outliers) %>% add_mutate_branch(femininity, strength, y) %>% add_formula_branch(model) %>% add_family_branch(distribution) multiverse_tree(mv) # Display a multiverse tree with branch conditions. match_poisson <- branch_condition(alldeaths, poisson) match_log_lin <- branch_condition(log(alldeaths + 1), gaussian) add_branch_condition(mv, match_poisson) add_branch_condition(mv, match_log_lin) multiverse_tree(mv) # You can adjust colour scale of the edges # using a ggraph::scale_edge_colour*() function. multiverse_tree(mv) + ggraph::scale_edge_colour_viridis( discrete = TRUE, labels = c("Distribution", "Model", "Strength", "Femininity", "Outliers", "y") ) # Display a multiverse tree for a subset of branches # with name label for each option. multiverse_tree(mv, branches = c("y", "distribution"), label = "name") # with code label for each option. multiverse_tree(mv, branches = c("y", "distribution"), label = "code") # adjusting size and orientation of the labels multiverse_tree(mv, branches = c("y", "distribution"), label = "name", label_size = 4, label_angle = 45) }
Create a new mutate branch.
mutate_branch(..., name = NULL)mutate_branch(..., name = NULL)
... |
branch definition expressions. |
name |
(optional) Name for the new variable. |
a mutate_branch object.
Other mutate branch functions:
add_mutate_branch()
# Define mutate branches. hurricane_strength <- mutate_branch( # damage vs. wind speed vs.pressure NDAM, HighestWindSpeed, Minpressure_Updated_2014, # Standardized versions scale(NDAM), scale(HighestWindSpeed), -scale(Minpressure_Updated_2014), ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength)# Define mutate branches. hurricane_strength <- mutate_branch( # damage vs. wind speed vs.pressure NDAM, HighestWindSpeed, Minpressure_Updated_2014, # Standardized versions scale(NDAM), scale(HighestWindSpeed), -scale(Minpressure_Updated_2014), ) # Create a mverse and add the branch. mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength)
mverse objectConstructs a new mverse object which extends
multiverse::multiverse object.
mverse(data) create_multiverse(data)mverse(data) create_multiverse(data)
data |
source dataframe. |
A mverse object with the source dataframe attached.
# Create a mverse object. mv <- mverse(hurricane) # create_multiverse() is an alias of mverse(). mv <- create_multiverse(hurricane)# Create a mverse object. mv <- mverse(hurricane) # create_multiverse() is an alias of mverse(). mv <- create_multiverse(hurricane)
*_branch objects.Print method for *_branch objects.
## S3 method for class 'branch' print(x, ...)## S3 method for class 'branch' print(x, ...)
x |
a |
... |
ignored. for compatibility only. |
No return value, called for printing only.
A dataset containing card counts between 2,053 soccer players playing in the first male divisions of England, Germany, France, and Spain in the 2012-2013 season and 3,147 referees that these players played under in professional matches. The dataset contains other covariates including 2 independent skin tone ratings per player. Each line represents a player-referee pair.
soccersoccer
A data frame with 146,028 rows and 26 variables:
short player ID
player name
player club
country of player club ( England, Germany, France, and Spain)
player birthday
player height (in cm)
player weight (in kg)
detailed player position
number of games in the player-referee dyad
victories in the player-referee dyad
ties in the player-referee dyad
losses in the player-referee dyad
goals scored by a player in the player-referee dyad
number of yellow cards player received from referee
number of yellow-red cards player received from referee
number of red cards player received from referee
skin rating of photo by rater 1 (5-point scale ranging from “very light skin” to “very dark skin”)
skin rating of photo by rater 2 (5-point scale ranging from “very light skin” to “very dark skin”)
unique referee ID number (referee name removed for anonymizing purposes)
unique referee country ID number (country name removed for anonymizing purposes)
mean implicit bias score (using the race IAT) for referee country, higher values correspond to faster white | good, black | bad associations
sample size for race IAT in that particular country
standard error for mean estimate of race IAT
mean explicit bias score (using a racial thermometer task) for referee country, higher values correspond to greater feelings of warmth toward whites versus blacks
sample size for explicit bias in that particular country
standard error for mean estimate of explicit bias measure
The skin colour of each player was rated by two independent raters,
rater1 and rater2, and the 5-point scale values were
scaled to 0 to 1 - i.e., 0, 0.25, 0.5, 0.75, 1.
Silberzahn R, Uhlmann EL, Martin DP, et al. Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science. 2018;1(3):337-356. doi:10.1177/2515245917747646
Returns a ggplot object that displays
the specification curve as proposed by (Simonsohn et al. 2020).
Note that the order of universes may not correspond to the order
in the summary table.
spec_curve( .spec_summary, label = "name", order_by = c("estimate", "is_significant"), colour_by = "is_significant", palette_common = NULL, pointsize = 2, linewidth = 0.5, spec_matrix_spacing = 10, theme_common = ggplot2::theme_minimal(), sep = "---" )spec_curve( .spec_summary, label = "name", order_by = c("estimate", "is_significant"), colour_by = "is_significant", palette_common = NULL, pointsize = 2, linewidth = 0.5, spec_matrix_spacing = 10, theme_common = ggplot2::theme_minimal(), sep = "---" )
.spec_summary |
A specification table created using
|
label |
If "name", uses the branch option names. If "code", display the codes used to define the branch options. |
order_by |
A character vector by which the curve is sorted. |
colour_by |
The name of the variable to colour the curve. |
palette_common |
A character vector of colours to match the values of
the variable |
pointsize |
Size of the points in the specification curve and the specification matrix. |
linewidth |
Width of confidence interval lines. |
spec_matrix_spacing |
A numeric for adjusting the specification matrix
spacing passed to |
theme_common |
A |
sep |
A string used internally to create the specification matrix. The string must be distinct from all branch names, option names, and option codes. Use a different value if any of them contains the default value. |
a ggplot object with the specification curve plot for
the estimates passed in the spec_summary().
Simonsohn U, Simmons JP, Nelson LD (2020). “Specification curve analysis.” Nature Human Behaviour, 4(11), 1208–1214. doi:10.1038/s41562-020-0912-z.
Other specification curve analysis:
spec_summary()
femininity <- mutate_branch( 1 * (MasFem > 6), 1 * (MasFem > mean(MasFem)) ) y <- mutate_branch(log(alldeaths + 1), alldeaths) intensity <- mutate_branch( Minpressure_Updated_2014, Category, NDAM, HighestWindSpeed ) model <- formula_branch( y ~ femininity, y ~ femininity * intensity ) family <- family_branch( gaussian, poisson ) match_poisson <- branch_condition(alldeaths, poisson) match_gaussian <- branch_condition(log(alldeaths + 1), gaussian) stable <- mverse(hurricane) %>% add_mutate_branch(y, femininity, intensity) %>% add_formula_branch(model) %>% add_family_branch(family) %>% add_branch_condition(match_poisson, match_gaussian) %>% glm_mverse() %>% spec_summary("femininity") # default behaviour spec_curve(stable) # coloring and sorting based on other variable stable %>% dplyr::mutate(colour_by = y_branch) %>% spec_curve(order_by = c("estimate", "colour_by"), colour_by = "colour_by") # Because the output is a \code{ggplot} object, you can # further modify the asethetics of the specification curve # using \code{ggplot2::theme()} and the specication matrix # using \code{ggupset::theme_combmatrix()} spec_curve(stable) + ggplot2::labs(y = "Estimates", colour = "Significant at 0.05 level", title = "Specification curve of femininity") + ggplot2::theme(legend.position = "bottom") + ggupset::theme_combmatrix( combmatrix.label.width = ggplot2::unit(c(25, 100, 0, 0), "pt") )femininity <- mutate_branch( 1 * (MasFem > 6), 1 * (MasFem > mean(MasFem)) ) y <- mutate_branch(log(alldeaths + 1), alldeaths) intensity <- mutate_branch( Minpressure_Updated_2014, Category, NDAM, HighestWindSpeed ) model <- formula_branch( y ~ femininity, y ~ femininity * intensity ) family <- family_branch( gaussian, poisson ) match_poisson <- branch_condition(alldeaths, poisson) match_gaussian <- branch_condition(log(alldeaths + 1), gaussian) stable <- mverse(hurricane) %>% add_mutate_branch(y, femininity, intensity) %>% add_formula_branch(model) %>% add_family_branch(family) %>% add_branch_condition(match_poisson, match_gaussian) %>% glm_mverse() %>% spec_summary("femininity") # default behaviour spec_curve(stable) # coloring and sorting based on other variable stable %>% dplyr::mutate(colour_by = y_branch) %>% spec_curve(order_by = c("estimate", "colour_by"), colour_by = "colour_by") # Because the output is a \code{ggplot} object, you can # further modify the asethetics of the specification curve # using \code{ggplot2::theme()} and the specication matrix # using \code{ggupset::theme_combmatrix()} spec_curve(stable) + ggplot2::labs(y = "Estimates", colour = "Significant at 0.05 level", title = "Specification curve of femininity") + ggplot2::theme(legend.position = "bottom") + ggupset::theme_combmatrix( combmatrix.label.width = ggplot2::unit(c(25, 100, 0, 0), "pt") )
Returns estimates for a selected variable across the multiverse along with
the universe specification information in a table. The resulting table
can be used for spe_curve().
spec_summary(.mverse, var, conf.int = TRUE, conf.level = 0.95)spec_summary(.mverse, var, conf.int = TRUE, conf.level = 0.95)
.mverse |
A |
var |
A character specifying the variable of interest. |
conf.int |
Whether the table should include confidence intervals. |
conf.level |
The confidence level for the confidence level and
|
A spec_summary object that includes estimates and
specification across the multiverse for the selected term(s). A boolean
column is_significant indicates whether p.value for the
universe is less than the specified significance level
(1 - conf.level).
Other specification curve analysis:
spec_curve()
femininity <- mutate_branch( 1 * (MasFem > 6), 1 * (MasFem > mean(MasFem)) ) intensity <- mutate_branch( Minpressure_Updated_2014, Category, NDAM, HighestWindSpeed ) model <- formula_branch( log(alldeaths + 1) ~ femininity, log(alldeaths + 1) ~ femininity * intensity ) mv <- mverse(hurricane) %>% add_mutate_branch(femininity) %>% add_mutate_branch(intensity) %>% add_formula_branch(model) %>% lm_mverse() spec_summary(mv, "femininity")femininity <- mutate_branch( 1 * (MasFem > 6), 1 * (MasFem > mean(MasFem)) ) intensity <- mutate_branch( Minpressure_Updated_2014, Category, NDAM, HighestWindSpeed ) model <- formula_branch( log(alldeaths + 1) ~ femininity, log(alldeaths + 1) ~ femininity * intensity ) mv <- mverse(hurricane) %>% add_mutate_branch(femininity) %>% add_mutate_branch(intensity) %>% add_formula_branch(model) %>% lm_mverse() spec_summary(mv, "femininity")
This method returns the multiverse table displaying all universes defined by the multiverse. Each row corresponds to a universe and the columns include universe number, branch option name, and branch option definition.
## S3 method for class 'mverse' summary(object, ...) ## S3 method for class 'lm_mverse' summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...) ## S3 method for class 'glm_mverse' summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...) ## S3 method for class 'glm.nb_mverse' summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...)## S3 method for class 'mverse' summary(object, ...) ## S3 method for class 'lm_mverse' summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...) ## S3 method for class 'glm_mverse' summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...) ## S3 method for class 'glm.nb_mverse' summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...)
object |
a |
... |
Ignored. |
conf.int |
When |
conf.level |
The confidence level of the confidence interval
returned using |
output |
The output of interest. The possible values are "estimates" ("e"), "df", "deviance" ("de"), and "aic" ("bic"). Alternatively, the first letters may be used. Default value is "estimates". |
When you pass a mverse objected fitted with model,
the summary table includes results of the fitted models
across the multiverse.
a multiverse table as a tibble.
# Displaying the multiverse table without any fitted values. hurricane_strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014 ) mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength) summary(mv) ## Displaying after adding a a filter branch. hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), !Name %in% c("Katrina"), TRUE # include all ) mv <- add_filter_branch(mv, hurricane_outliers) summary(mv) # Displaying the multiverse table with \code{lm} models fitted. hurricane_strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014 ) y <- mutate_branch( alldeaths, log(alldeaths + 1) ) hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch( y ~ MasFem, y ~ MasFem + hurricane_strength ) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_mutate_branch(hurricane_strength, y) %>% add_formula_branch(model_specifications) %>% lm_mverse() summary(mv) # Displaying the multiverse table with \code{glm} models fitted. hurricane_strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014 ) hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch( alldeaths ~ MasFem, alldeaths ~ MasFem + hurricane_strength ) model_distributions <- family_branch(poisson) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_mutate_branch(hurricane_strength) %>% add_formula_branch(model_specifications) %>% add_family_branch(model_distributions) %>% glm_mverse() summary(mv) # Displaying the multiverse table with \code{glm.nb} models fitted. hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch(alldeaths ~ MasFem) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_formula_branch(model_specifications) %>% glm.nb_mverse() summary(mv)# Displaying the multiverse table without any fitted values. hurricane_strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014 ) mv <- create_multiverse(hurricane) %>% add_mutate_branch(hurricane_strength) summary(mv) ## Displaying after adding a a filter branch. hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), !Name %in% c("Katrina"), TRUE # include all ) mv <- add_filter_branch(mv, hurricane_outliers) summary(mv) # Displaying the multiverse table with \code{lm} models fitted. hurricane_strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014 ) y <- mutate_branch( alldeaths, log(alldeaths + 1) ) hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch( y ~ MasFem, y ~ MasFem + hurricane_strength ) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_mutate_branch(hurricane_strength, y) %>% add_formula_branch(model_specifications) %>% lm_mverse() summary(mv) # Displaying the multiverse table with \code{glm} models fitted. hurricane_strength <- mutate_branch( NDAM, HighestWindSpeed, Minpressure_Updated_2014 ) hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch( alldeaths ~ MasFem, alldeaths ~ MasFem + hurricane_strength ) model_distributions <- family_branch(poisson) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_mutate_branch(hurricane_strength) %>% add_formula_branch(model_specifications) %>% add_family_branch(model_distributions) %>% glm_mverse() summary(mv) # Displaying the multiverse table with \code{glm.nb} models fitted. hurricane_outliers <- filter_branch( !Name %in% c("Katrina", "Audrey", "Andrew"), TRUE # include all ) model_specifications <- formula_branch(alldeaths ~ MasFem) mv <- create_multiverse(hurricane) %>% add_filter_branch(hurricane_outliers) %>% add_formula_branch(model_specifications) %>% glm.nb_mverse() summary(mv)
t_test_mverse performs t-tests across the multiverse.
If x or y is specified, then performs one and two sample t-tests
on specified columns of the data. If both x and y are NULL, then
performs t.test based on the formula branches.
t_test_mverse( .mverse, x = NULL, y = NULL, alternative = "two.sided", mu = 0, paired = FALSE, var.equal = FALSE, conf.level = 0.95, parallel = FALSE, progress = FALSE )t_test_mverse( .mverse, x = NULL, y = NULL, alternative = "two.sided", mu = 0, paired = FALSE, var.equal = FALSE, conf.level = 0.95, parallel = FALSE, progress = FALSE )
.mverse |
a |
x |
(optional) column name of data within mverse object |
y |
(optional) column name of data within mverse object |
alternative |
a character string specifying the alternative hypothesis, must be one of "two.sided" (default), "greater" or "less". You can specify just the initial letter. |
mu |
a number indicating the true value of the mean (or difference in means if you are performing a two sample test). |
paired |
a logical indicating whether you want a paired t-test. |
var.equal |
a logical variable indicating whether to treat the two variances as being equal. |
conf.level |
confidence level of the interval. |
parallel |
passed to |
progress |
passed to |
a multiverse table displaying the t-test results as a tibble.
# Performing a unpaired two sample t-test. library(dplyr) mv <- soccer %>% filter(!is.na(rater1), !is.na(rater2)) %>% mverse() x <- mutate_branch( ((rater1 + rater2) / 2) > mean((rater1 + rater2) / 2), ifelse(rater1 > rater2, rater1 > 0.5, rater2 > 0.5) ) y <- mutate_branch( redCards, yellowCards, yellowReds ) two_sample_form <- formula_branch(y ~ x) mv <- mv %>% add_mutate_branch(x, y) %>% add_formula_branch(two_sample_form) t_test_mverse(mv)# Performing a unpaired two sample t-test. library(dplyr) mv <- soccer %>% filter(!is.na(rater1), !is.na(rater2)) %>% mverse() x <- mutate_branch( ((rater1 + rater2) / 2) > mean((rater1 + rater2) / 2), ifelse(rater1 > rater2, rater1 > 0.5, rater2 > 0.5) ) y <- mutate_branch( redCards, yellowCards, yellowReds ) two_sample_form <- formula_branch(y ~ x) mv <- mv %>% add_mutate_branch(x, y) %>% add_formula_branch(two_sample_form) t_test_mverse(mv)