Package 'POMADE' reference manual

Title:	Power for Meta-Analysis of Dependent Effects
Description:	Provides functions to compute and plot power levels, minimum detectable effect sizes, and minimum required sample sizes for the test of the overall average effect size in meta-analysis of dependent effect sizes.
Authors:	Mikkel H. Vembye [aut, cre] , James E. Pustejovsky [aut]
Maintainer:	Mikkel H. Vembye <[email protected]>
License:	MIT + file LICENSE
Version:	0.2.0.9999
Built:	2025-02-09 05:30:54 UTC
Source:	https://github.com/mikkelvembye/pomade

Cluster Bias Correction

Description

Function to conduct cluster bias correction of sampling variance estimates obtained from cluster-randomized studies in which the reported variance does not account for clustering.

Usage

cluster_bias_adjustment(sigma2js, cluster_size = 22, icc = 0.2)
cluster_bias_adjustment(sigma2js, cluster_size = 22, icc = 0.2)

Arguments

`sigma2js`	A vector of sampling variance estimates that do not account for clustering.
`cluster_size`	A numerical value for average cluster size.
`icc`	Assumed intra-class correlation (proportion of total variance at the cluster level).

Value

Returns a vector of cluster bias adjusted variance estimates

Examples

cbc_var <- cluster_bias_adjustment(
  sigma2js = c(0.04, 0.06, 0.08, 0.1),
  cluster_size = 15,
  icc = 0.15
)

cbc_var


cbc_var <- cluster_bias_adjustment(
  sigma2js = c(0.04, 0.06, 0.08, 0.1),
  cluster_size = 15,
  icc = 0.15
)

cbc_var

Approximate Effective Sample Sizes

Description

Approximate Effective Sample Sizes

Usage

effective_sample_sizes(
  sample_sizes_raw = NULL,
  Nt_raw = NULL,
  Nc_raw = NULL,
  cluster_size = 22,
  icc = 0.22
)
effective_sample_sizes(
  sample_sizes_raw = NULL,
  Nt_raw = NULL,
  Nc_raw = NULL,
  cluster_size = 22,
  icc = 0.22
)

Arguments

`sample_sizes_raw`	Vector of the raw total study sample size(s).
`Nt_raw`	Vector of raw treatment group sample size(s).
`Nc_raw`	Vector of raw control group sample size(s).
`cluster_size`	Average cluster size (Default = 22, a common class size in education research studies).
`icc`	Assumed intra-class correlation (Default = 0.22, the average ICC value in Hedges & Hedberg (2007) unconditional models)

Details

N_j/DE

Value

A vector of effective sample sizes, adjusted for cluster-dependence.

Examples

sample_sizes <- sample(50:1000, 50, replace = TRUE)
effective_sample_sizes(
  sample_sizes_raw = sample_sizes,
  cluster_size = 20,
  icc = 0.15
)


sample_sizes <- sample(50:1000, 50, replace = TRUE)
effective_sample_sizes(
  sample_sizes_raw = sample_sizes,
  cluster_size = 20,
  icc = 0.15
)

Minimum Detectable Effect Size (MDES) for Meta-Analysis With Dependent Effect Sizes

Description

Compute the minimum detectable effect size in a meta-analysis of dependent effect size estimates, given a specified number of studies, power level, estimation method, and further assumptions about the distribution of studies.

Usage

mdes_MADE(
  J,
  tau,
  omega,
  rho,
  alpha = 0.05,
  target_power = 0.8,
  d = 0,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = NULL,
  n_ES_dist = NULL,
  iterations = 100,
  seed = NULL,
  warning = TRUE,
  upper = 2,
  show_lower = FALSE
)
mdes_MADE(
  J,
  tau,
  omega,
  rho,
  alpha = 0.05,
  target_power = 0.8,
  d = 0,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = NULL,
  n_ES_dist = NULL,
  iterations = 100,
  seed = NULL,
  warning = TRUE,
  upper = 2,
  show_lower = FALSE
)

Arguments

`J`	Number of studies. Can be one value or a vector of multiple values.
`tau`	Between-study SD. Can be one value or a vector of multiple values.
`omega`	Within-study SD. Can be one value or a vector of multiple values.
`rho`	Correlation coefficient between effect size estimates from the same study. Can be one value or a vector of multiple values.
`alpha`	Level of statistical significance. Can be one value or a vector of multiple values. Default is 0.05.
`target_power`	Numerical value specifying the target power level. Can be one value or a vector of multiple values.
`d`	Contrast value. Can be one value or a vector of multiple values. Default is 0.
`model`	Assumed working model for dependent effect sizes, either `"CHE"` for the correlated-and-hierarchical effects model, `"CE"` for the correlated effects model, or `"MLMA"` for the multi-level meta-analysis model. Default is `"CHE"`. Can be one value or a vector of multiple values.
`var_df`	Indicates the technique used to obtain the sampling variance of the average effect size estimate and the degrees of freedom, either `"Model"` for model-based variance estimator with degrees of freedom of `J - 1`, `"Satt"` for model-based variance estimator with Satterthwaite degrees of freedom, or `"RVE"` for robust variance estimator with Satterthwaite degrees of freedom. Default is `"RVE"`. Can be one value or a vector of multiple values.
`sigma2_dist`	Distribution of sampling variance estimates from each study. Can be either a single value, a vector of plausible values, or a function that generates random values.
`n_ES_dist`	Distribution of the number of effect sizes per study. Can be either a single value, a vector of plausible values, or a function that generates random values.
`iterations`	Number of iterations per condition (default is 100).
`seed`	Numerical value for a seed to ensure reproducibility of the iterated power approximations.
`warning`	Logical indicating whether to return a warning when either sigma2_dist or n_ES_dist is based on balanced assumptions.
`upper`	Numerical value containing the upper bound of the interval to be searched for the MDES.
`show_lower`	Logical value indicating whether to report lower bound of the interval searched for the MDES. Default is `FALSE`.

Value

Returns a tibble with information about the expectation of the number of studies, the between-study and within-study variance components, the sample correlation, the contrast effect, the level of statistical significance, the target power value(s), the minimum detectable effect size, the number of iterations, the model to handle dependent effect sizes, and the methods used to obtain sampling variance estimates as well as the number effect sizes per study.

Examples


mdes_MADE(
  J = 30,
  tau = 0.05,
  omega = 0.02,
  rho = 0.2,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = 4 / 100,
  n_ES_dist = 6,
  seed = 10052510
)


mdes_MADE(
  J = 30,
  tau = 0.05,
  omega = 0.02,
  rho = 0.2,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = 4 / 100,
  n_ES_dist = 6,
  seed = 10052510
)

Finding the Number of Studies Needed to Obtain a Certain Amount of Power

Description

Compute the minimum number of studies needed to obtain a specified power level in a meta-analysis of dependent effect size estimates, given an effect size of practical concern, estimation method, and further assumptions about the distribution of studies.

Usage

min_studies_MADE(
  mu,
  tau,
  omega,
  rho,
  alpha = 0.05,
  target_power = 0.8,
  d = 0,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = NULL,
  n_ES_dist = NULL,
  iterations = 100,
  seed = NULL,
  warning = TRUE,
  upper = 100,
  show_lower = FALSE
)
min_studies_MADE(
  mu,
  tau,
  omega,
  rho,
  alpha = 0.05,
  target_power = 0.8,
  d = 0,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = NULL,
  n_ES_dist = NULL,
  iterations = 100,
  seed = NULL,
  warning = TRUE,
  upper = 100,
  show_lower = FALSE
)

Arguments

`mu`	Effect size of practical concern. Can be one value or a vector of multiple values.
`tau`	Between-study SD. Can be one value or a vector of multiple values.
`omega`	Within-study SD. Can be one value or a vector of multiple values.
`rho`	Correlation coefficient between effect size estimates from the same study. Can be one value or a vector of multiple values.
`alpha`	Level of statistical significance. Can be one value or a vector of multiple values. Default is 0.05.
`target_power`	Numerical value specifying the target power level. Can be one value or a vector of multiple values.
`d`	Contrast value. Can be one value or a vector of multiple values. Default is 0.
`model`	Assumed working model for dependent effect sizes, either `"CHE"` for the correlated-and-hierarchical effects model, `"CE"` for the correlated effects model, or `"MLMA"` for the multi-level meta-analysis model. Default is `"CHE"`. Can be one value or a vector of multiple values.
`var_df`	Indicates the technique used to obtain the sampling variance of the average effect size estimate and the degrees of freedom, either `"Model"` for model-based variance estimator with degrees of freedom of `J - 1`, `"Satt"` for model-based variance estimator with Satterthwaite degrees of freedom, or `"RVE"` for robust variance estimator with Satterthwaite degrees of freedom. Default is `"RVE"`. Can be one value or a vector of multiple values.
`sigma2_dist`	Distribution of sampling variance estimates from each study. Can be either a single value, a vector of plausible values, or a function that generates random values.
`n_ES_dist`	Distribution of the number of effect sizes per study. Can be either a single value, a vector of plausible values, or a function that generates random values.
`iterations`	Number of iterations per condition (default is 100).
`seed`	Numerical value for a seed to ensure reproducibility of the iterated power approximations.
`warning`	Logical indicating whether to return a warning when either sigma2_dist or n_ES_dist is based on balanced assumptions.
`upper`	Numerical value containing the upper bound of the interval to be searched for the minimum number of studies.
`show_lower`	Logical value indicating whether to report lower bound of the interval searched for the minimum number of studies. Default is `FALSE`.

Value

Returns a tibble with information about the expectation of the effect size of practical concern, the between-study and within-study variance components, the sample correlation, the contrast effect, the level of statistical significance, the target power value(s), the number of studies needed, the number of iterations, the model to handle dependent effect sizes, and the methods used to obtain sampling variance estimates as well as the number effect sizes per study.

Examples


min_studies_MADE(
  mu = 0.3,
  tau = 0.05,
  omega = 0.01,
  rho = 0.2,
  target_power = .7,
  alpha = 0.05,
  model = "CE",
  var_df = "RVE",
  sigma2_dist = 4 / 200,
  n_ES_dist = 5.5,
  seed = 10052510
)


min_studies_MADE(
  mu = 0.3,
  tau = 0.05,
  omega = 0.01,
  rho = 0.2,
  target_power = .7,
  alpha = 0.05,
  model = "CE",
  var_df = "RVE",
  sigma2_dist = 4 / 200,
  n_ES_dist = 5.5,
  seed = 10052510
)

Generic plot function for 'MADE' objects

Description

Create a faceted plot displaying the results of a set of power analyses. This is a generic function to make facet_grid plots, with specific methods defined for power_MADE, mdes_MADE, and min_studies_MADE objects.

Usage

plot_MADE(
  data,
  v_lines,
  legend_position,
  color,
  numbers,
  number_size,
  numbers_ynudge,
  caption,
  x_lab,
  x_breaks,
  x_limits,
  y_breaks,
  y_limits,
  y_expand = NULL,
  warning = TRUE,
  traffic_light_assumptions = NULL,
  traffic_light_palette = "green-yellow-red",
  ...
)
plot_MADE(
  data,
  v_lines,
  legend_position,
  color,
  numbers,
  number_size,
  numbers_ynudge,
  caption,
  x_lab,
  x_breaks,
  x_limits,
  y_breaks,
  y_limits,
  y_expand = NULL,
  warning = TRUE,
  traffic_light_assumptions = NULL,
  traffic_light_palette = "green-yellow-red",
  ...
)

Arguments

`data`	Data/object for which the plot should be made.
`v_lines`	Integer or vector to specify vertical line(s) in within each plot. Default is `NULL`.
`legend_position`	Character string to specify position of legend. Default is `"bottom"`.
`color`	Logical indicating whether to use color in the plot(s). Default is `TRUE`.
`numbers`	Logical indicating whether to number the plots. Default is `TRUE`.
`number_size`	Integer value specifying the size of the (optional) plot numbers. Default is `2.5`.
`numbers_ynudge`	Integer value for vertical nudge of the (optional) plot numbers.
`caption`	Logical indicating whether to include a caption with detailed information regarding the analysis. Default is `TRUE`.
`x_lab`	Title for the x-axis. If `NULL` (the default), the x_lab is specified automatically.
`x_breaks`	Optional vector to specify breaks on the x-axis. Default is `NULL`.
`x_limits`	Optional vector of length 2 to specify the limits of the x-axis. Default is `NULL`, which allows limits to be determined automatically from the data.
`y_breaks`	Optional vector to specify breaks on the y-axis.
`y_limits`	Optional vector of length 2 to specify the limits of the y-axis.
`y_expand`	Optional vector to expand the limits of the y-axis. Default is `NULL`.
`warning`	Logical indicating whether warnings should be returned when multiple models appear in the data. Default is `TRUE`.
`traffic_light_assumptions`	Optional logical to specify coloring of strips of the facet grids to emphasize assumptions about the likelihood the given analytical scenario. See Vembye, Pustejovsky, & Pigott (forthcoming) for further details.
`traffic_light_palette`	Character string or character vector to control the color of traffic light strips. If set to `'green-yellow-red'` (the default), expected scenarios will be colored green, likely scenarios will be colored yellow, and unlikely scenarios will be colored red. If set to `'greyscale'`, a gray-scale version of the traffic light plot is provided with white indicating the expected scenario, light gray indicating other plausible scenarios, and dark gray indicating less likely scenarios. Users can also specify their own palette by setting `traffic_light_palette` to a character vector with colors for the expected, likely, and unlikely scenarios. In this case, the vector must have three entries named `'expected'`, `'likely'`, and `'unlikely'`,
`...`	Additional arguments available for some classes of objects.

Value

A ggplot object

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (In preparation). Conducting power analysis for meta-analysis of dependent effect sizes: Common guidelines and an introduction to the POMADE R package.

Examples

power_dat <-
  power_MADE(
    J = c(50, 56),
    mu = 0.15,
    tau = 0.1,
    omega = 0.05,
    rho = 0,
    sigma2_dist = 4 / 200,
    n_ES_dist = 6
  )

power_example <-
  plot_MADE(
   data = power_dat,
   power_min = 0.8,
   expected_studies = c(52, 54),
   warning = FALSE,
   caption = TRUE,
   color = TRUE,
   model_comparison = FALSE,
   numbers = FALSE
   )

power_example


power_dat <-
  power_MADE(
    J = c(50, 56),
    mu = 0.15,
    tau = 0.1,
    omega = 0.05,
    rho = 0,
    sigma2_dist = 4 / 200,
    n_ES_dist = 6
  )

power_example <-
  plot_MADE(
   data = power_dat,
   power_min = 0.8,
   expected_studies = c(52, 54),
   warning = FALSE,
   caption = TRUE,
   color = TRUE,
   model_comparison = FALSE,
   numbers = FALSE
   )

power_example

Plot function for a 'mdes' object

Description

Creates a faceted plot for minimum detectable effect size (mdes) analyses calculated using mdes_MADE.

Usage

## S3 method for class 'mdes'
plot_MADE(
  data,
  v_lines = NULL,
  legend_position = "bottom",
  color = TRUE,
  numbers = TRUE,
  number_size = 2.5,
  numbers_ynudge = NULL,
  caption = TRUE,
  x_lab = NULL,
  x_breaks = NULL,
  x_limits = NULL,
  y_breaks = ggplot2::waiver(),
  y_limits = NULL,
  y_expand = NULL,
  warning = TRUE,
  traffic_light_assumptions = NULL,
  traffic_light_palette = "green-yellow-red",
  es_min = NULL,
  expected_studies = NULL,
  ...
)
## S3 method for class 'mdes'
plot_MADE(
  data,
  v_lines = NULL,
  legend_position = "bottom",
  color = TRUE,
  numbers = TRUE,
  number_size = 2.5,
  numbers_ynudge = NULL,
  caption = TRUE,
  x_lab = NULL,
  x_breaks = NULL,
  x_limits = NULL,
  y_breaks = ggplot2::waiver(),
  y_limits = NULL,
  y_expand = NULL,
  warning = TRUE,
  traffic_light_assumptions = NULL,
  traffic_light_palette = "green-yellow-red",
  es_min = NULL,
  expected_studies = NULL,
  ...
)

Arguments

`data`	Data/object for which the plot should be made.
`v_lines`	Integer or vector to specify vertical line(s) in within each plot. Default is `NULL`.
`legend_position`	Character string to specify position of legend. Default is `"bottom"`.
`color`	Logical indicating whether to use color in the plot(s). Default is `TRUE`.
`numbers`	Logical indicating whether to number the plots. Default is `TRUE`.
`number_size`	Integer value specifying the size of the (optional) plot numbers. Default is `2.5`.
`numbers_ynudge`	Integer value for vertical nudge of the (optional) plot numbers.
`caption`	Logical indicating whether to include a caption with detailed information regarding the analysis. Default is `TRUE`.
`x_lab`	Title for the x-axis. If `NULL` (the default), the x_lab is specified automatically.
`x_breaks`	Optional vector to specify breaks on the x-axis. Default is `NULL`.
`x_limits`	Optional vector of length 2 to specify the limits of the x-axis. Default is `NULL`, which allows limits to be determined automatically from the data.
`y_breaks`	Optional vector to specify breaks on the y-axis.
`y_limits`	Optional vector of length 2 to specify the limits of the y-axis.
`y_expand`	Optional vector to expand the limits of the y-axis. Default is `NULL`.
`warning`	Logical indicating whether warnings should be returned when multiple models appear in the data. Default is `TRUE`.
`traffic_light_assumptions`	Optional logical to specify coloring of strips of the facet grids to emphasize assumptions about the likelihood the given analytical scenario. See Vembye, Pustejovsky, & Pigott (forthcoming) for further details.
`traffic_light_palette`	Character string or character vector to control the color of traffic light strips. If set to `'green-yellow-red'` (the default), expected scenarios will be colored green, likely scenarios will be colored yellow, and unlikely scenarios will be colored red. If set to `'greyscale'`, a gray-scale version of the traffic light plot is provided with white indicating the expected scenario, light gray indicating other plausible scenarios, and dark gray indicating less likely scenarios. Users can also specify their own palette by setting `traffic_light_palette` to a character vector with colors for the expected, likely, and unlikely scenarios. In this case, the vector must have three entries named `'expected'`, `'likely'`, and `'unlikely'`,
`es_min`	Optional integer or vector to specify a horizontal line or interval, indicating a benchmark value or values for the minimum effect size of practical concern (default is `NULL`).
`expected_studies`	Optional vector of length 2 specifying a range for the number of studies one expects to include in the meta-analysis. If specified, this interval will be shaded across facet_grip plots (default is `NULL`).
`...`	Additional arguments available for some classes of objects.

Details

In general, it can be rather difficult to guess/approximate the true model parameters and sample characteristics a priori. Calculating the minimum detectable effect size under just a single set of assumptions can easily be misleading even if the true model and data structure only slightly diverge from the yielded data and model assumptions. To maximize the informativeness of the analysis, Vembye, Pustejovsky, & Pigott (In preparation) suggest accommodating the uncertainty of the power approximations by reporting or plotting minimum detectable effect size estimates across a range of possible scenarios, which can be done using plot_MADE.mdes.

Value

A ggplot plot showing the minimum detectable effect size across the expected number of studies, faceted by the between-study and within-study SDs, with different colors, lines, and shapes corresponding to different values of the assumed sample correlation.

References

Examples

mdes_MADE(
  J = c(25, 35),
  tau = 0.05,
  omega = 0,
  rho = 0,
  target_power = .6,
  alpha = 0.1,
  sigma2_dist = 4 / 200,
  n_ES_dist = 8,
  seed = 10052510
) |>
  plot_MADE(expected_studies = c(28, 32), numbers = FALSE)



mdes_MADE(
  J = c(25, 35),
  tau = 0.05,
  omega = 0,
  rho = 0,
  target_power = .6,
  alpha = 0.1,
  sigma2_dist = 4 / 200,
  n_ES_dist = 8,
  seed = 10052510
) |>
  plot_MADE(expected_studies = c(28, 32), numbers = FALSE)

Plot function for a 'min_studies' object

Description

Creates a faceted plot with analyses of the minimum number of studies needed to obtained a given effect size with specified levels of power, as calculated using min_studies_MADE.

Usage

## S3 method for class 'min_studies'
plot_MADE(
  data,
  v_lines = NULL,
  legend_position = "bottom",
  color = TRUE,
  numbers = TRUE,
  number_size = 2.5,
  numbers_ynudge = NULL,
  caption = TRUE,
  x_lab = NULL,
  x_breaks = NULL,
  x_limits = NULL,
  y_breaks = ggplot2::waiver(),
  y_limits = NULL,
  y_expand = NULL,
  warning = TRUE,
  traffic_light_assumptions = NULL,
  traffic_light_palette = "green-yellow-red",
  v_shade = NULL,
  h_lines = NULL,
  ...
)
## S3 method for class 'min_studies'
plot_MADE(
  data,
  v_lines = NULL,
  legend_position = "bottom",
  color = TRUE,
  numbers = TRUE,
  number_size = 2.5,
  numbers_ynudge = NULL,
  caption = TRUE,
  x_lab = NULL,
  x_breaks = NULL,
  x_limits = NULL,
  y_breaks = ggplot2::waiver(),
  y_limits = NULL,
  y_expand = NULL,
  warning = TRUE,
  traffic_light_assumptions = NULL,
  traffic_light_palette = "green-yellow-red",
  v_shade = NULL,
  h_lines = NULL,
  ...
)

Arguments

`data`	Data/object for which the plot should be made.
`v_lines`	Integer or vector to specify vertical line(s) in within each plot. Default is `NULL`.
`legend_position`	Character string to specify position of legend. Default is `"bottom"`.
`color`	Logical indicating whether to use color in the plot(s). Default is `TRUE`.
`numbers`	Logical indicating whether to number the plots. Default is `TRUE`.
`number_size`	Integer value specifying the size of the (optional) plot numbers. Default is `2.5`.
`numbers_ynudge`	Integer value for vertical nudge of the (optional) plot numbers.
`caption`	Logical indicating whether to include a caption with detailed information regarding the analysis. Default is `TRUE`.
`x_lab`	Title for the x-axis. If `NULL` (the default), the x_lab is specified automatically.
`x_breaks`	Optional vector to specify breaks on the x-axis. Default is `NULL`.
`x_limits`	Optional vector of length 2 to specify the limits of the x-axis. Default is `NULL`, which allows limits to be determined automatically from the data.
`y_breaks`	Optional vector to specify breaks on the y-axis.
`y_limits`	Optional vector of length 2 to specify the limits of the y-axis.
`y_expand`	Optional vector to expand the limits of the y-axis. Default is `NULL`.
`warning`	Logical indicating whether warnings should be returned when multiple models appear in the data. Default is `TRUE`.
`traffic_light_assumptions`	Optional logical to specify coloring of strips of the facet grids to emphasize assumptions about the likelihood the given analytical scenario. See Vembye, Pustejovsky, & Pigott (forthcoming) for further details.
`traffic_light_palette`	Character string or character vector to control the color of traffic light strips. If set to `'green-yellow-red'` (the default), expected scenarios will be colored green, likely scenarios will be colored yellow, and unlikely scenarios will be colored red. If set to `'greyscale'`, a gray-scale version of the traffic light plot is provided with white indicating the expected scenario, light gray indicating other plausible scenarios, and dark gray indicating less likely scenarios. Users can also specify their own palette by setting `traffic_light_palette` to a character vector with colors for the expected, likely, and unlikely scenarios. In this case, the vector must have three entries named `'expected'`, `'likely'`, and `'unlikely'`,
`v_shade`	Optional vector of length 2 specifying the range of the x-axis interval to be shaded in each plot.
`h_lines`	Optional integer or vector specifying horizontal lines on each plot.
`...`	Additional arguments available for some classes of objects.

Details

In general, it can be rather difficult to guess/approximate the true model parameters and sample characteristics a priori. Calculating the minimum number of studies needed under just a single set of assumptions can easily be misleading even if the true model and data structure only slightly diverge from the yielded data and model assumptions. To maximize the informativeness of the analysis, Vembye, Pustejovsky, & Pigott (In preparation) suggest accommodating the uncertainty of the power approximations by reporting or plotting power estimates across a range of possible scenarios, which can be done using plot_MADE.power.

Value

A ggplot plot showing the minimum number of studies needed to obtain a given effect size with a certain amount of power and level-alpha, faceted across levels of the within-study SD and the between-study SD, with different colors, lines, and shapes corresponding to different values of the assumed sample correlation. If length(unique(data$mu)) > 1, it returns a ggplot plot showing the minimum studies needed to obtained a given effect size with a certain amount of power and level-alpha across effect sizes of practical concern, faceted by the between-study and within-study SDs, with different colors, lines, and shapes corresponding to different values of the assumed sample correlation.

References

Examples

min_studies_MADE(
  mu = c(0.25, 0.35),
  tau = 0.05,
  omega = 0.02,
  rho = 0.2,
  target_power = .7,
  sigma2_dist = 4 / 200,
  n_ES_dist = 6,
  seed = 10052510
) |>
  plot_MADE(y_breaks = seq(0, 10, 2), numbers = FALSE)

min_studies_MADE(
  mu = c(0.25, 0.35),
  tau = 0.05,
  omega = 0.02,
  rho = 0.2,
  target_power = .7,
  sigma2_dist = 4 / 200,
  n_ES_dist = 6,
  seed = 10052510
) |>
  plot_MADE(y_breaks = seq(0, 10, 2), numbers = FALSE)

Plot function for a 'power' object

Description

Creates a faceted plot or plots for power analyses conducted with power_MADE.

Usage

## S3 method for class 'power'
plot_MADE(
  data,
  v_lines = NULL,
  legend_position = "bottom",
  color = TRUE,
  numbers = TRUE,
  number_size = 2.5,
  numbers_ynudge = 0,
  caption = TRUE,
  x_lab = NULL,
  x_breaks = NULL,
  x_limits = NULL,
  y_breaks = seq(0, 1, 0.2),
  y_limits = c(0, 1),
  y_expand = NULL,
  warning = TRUE,
  traffic_light_assumptions = NULL,
  traffic_light_palette = "green-yellow-red",
  power_min = NULL,
  expected_studies = NULL,
  model_comparison = FALSE,
  ...
)
## S3 method for class 'power'
plot_MADE(
  data,
  v_lines = NULL,
  legend_position = "bottom",
  color = TRUE,
  numbers = TRUE,
  number_size = 2.5,
  numbers_ynudge = 0,
  caption = TRUE,
  x_lab = NULL,
  x_breaks = NULL,
  x_limits = NULL,
  y_breaks = seq(0, 1, 0.2),
  y_limits = c(0, 1),
  y_expand = NULL,
  warning = TRUE,
  traffic_light_assumptions = NULL,
  traffic_light_palette = "green-yellow-red",
  power_min = NULL,
  expected_studies = NULL,
  model_comparison = FALSE,
  ...
)

Arguments

`data`	Data/object for which the plot should be made.
`v_lines`	Integer or vector to specify vertical line(s) in within each plot. Default is `NULL`.
`legend_position`	Character string to specify position of legend. Default is `"bottom"`.
`color`	Logical indicating whether to use color in the plot(s). Default is `TRUE`.
`numbers`	Logical indicating whether to number the plots. Default is `TRUE`.
`number_size`	Integer value specifying the size of the (optional) plot numbers. Default is `2.5`.
`numbers_ynudge`	Integer value for vertical nudge of the (optional) plot numbers.
`caption`	Logical indicating whether to include a caption with detailed information regarding the analysis. Default is `TRUE`.
`x_lab`	Title for the x-axis. If `NULL` (the default), the x_lab is specified automatically.
`x_breaks`	Optional vector to specify breaks on the x-axis. Default is `NULL`.
`x_limits`	Optional vector of length 2 to specify the limits of the x-axis. Default is `NULL`, which allows limits to be determined automatically from the data.
`y_breaks`	Optional vector to specify breaks on the y-axis.
`y_limits`	Optional vector of length 2 to specify the limits of the y-axis.
`y_expand`	Optional vector to expand the limits of the y-axis. Default is `NULL`.
`warning`	Logical indicating whether warnings should be returned when multiple models appear in the data. Default is `TRUE`.
`traffic_light_assumptions`	Optional logical to specify coloring of strips of the facet grids to emphasize assumptions about the likelihood the given analytical scenario. See Vembye, Pustejovsky, & Pigott (forthcoming) for further details.
`traffic_light_palette`	Character string or character vector to control the color of traffic light strips. If set to `'green-yellow-red'` (the default), expected scenarios will be colored green, likely scenarios will be colored yellow, and unlikely scenarios will be colored red. If set to `'greyscale'`, a gray-scale version of the traffic light plot is provided with white indicating the expected scenario, light gray indicating other plausible scenarios, and dark gray indicating less likely scenarios. Users can also specify their own palette by setting `traffic_light_palette` to a character vector with colors for the expected, likely, and unlikely scenarios. In this case, the vector must have three entries named `'expected'`, `'likely'`, and `'unlikely'`,
`power_min`	Either an integer specify a horizontal line or a length-2 vector to specify an interval, indicating a benchmark level of power (default is `NULL`).
`expected_studies`	Optional vector of length 2 specifying a range for the number of studies one expects to include in the meta-analysis. If specified, this interval will be shaded across facet_grip plots (default is `NULL`).
`model_comparison`	Logical indicating whether power estimates should be plotted across different working models for dependent effect size estimates (default is `FALSE`) instead of across values for the sampling correlation.
`...`	Additional arguments available for some classes of objects.

Details

In general, it can be rather difficult to guess/approximate the true model parameters and sample characteristics a priori. Calculating power under only a single set of assumptions can easily be misleading even if the true model and data structure only slightly diverge from the yielded data and model assumptions. To maximize the informativeness of the power approximations, Vembye, Pustejovsky, & Pigott (In preparation) suggest accommodating the uncertainty of the power approximations by reporting or plotting power estimates across a range of possible scenarios, which can be done using plot_MADE.power.

Value

A ggplot plot showing power across the expected number of studies, faceted by the between-study and within-study SDs, with different colors, lines, and shapes corresponding to different values of the assumed sample correlation. If model_comparison = TRUE, it returns a ggplot plot showing power across the expected number of studies, faceted by the between-study and within-study SDs, with different colors, lines, and shapes corresponding to different working models for dependent effect size estimates

References

Examples

power_dat <-
  power_MADE(
    J = c(50, 56),
    mu = 0.15,
    tau = 0.1,
    omega = 0.05,
    rho = 0,
    sigma2_dist = 4 / 200,
    n_ES_dist = 6
  )

power_example <-
  plot_MADE(
   data = power_dat,
   power_min = 0.8,
   expected_studies = c(52, 54),
   warning = FALSE,
   caption = TRUE,
   color = TRUE,
   model_comparison = FALSE,
   numbers = FALSE
   )

power_example


power_dat <-
  power_MADE(
    J = c(50, 56),
    mu = 0.15,
    tau = 0.1,
    omega = 0.05,
    rho = 0,
    sigma2_dist = 4 / 200,
    n_ES_dist = 6
  )

power_example <-
  plot_MADE(
   data = power_dat,
   power_min = 0.8,
   expected_studies = c(52, 54),
   warning = FALSE,
   caption = TRUE,
   color = TRUE,
   model_comparison = FALSE,
   numbers = FALSE
   )

power_example

Power Approximation for Overall Average Effects in Meta-Analysis With Dependent Effect Sizes

Description

Compute power of the test of the overall average effect size in a meta-analysis of dependent effect size estimates, given a specified number of studies, effect size of practical concern, estimation method, and further assumptions about the distribution of studies.

Usage

power_MADE(
  J,
  mu,
  tau,
  omega,
  rho,
  alpha = 0.05,
  d = 0,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = NULL,
  n_ES_dist = NULL,
  iterations = 100,
  seed = NULL,
  warning = TRUE,
  average_power = TRUE
)
power_MADE(
  J,
  mu,
  tau,
  omega,
  rho,
  alpha = 0.05,
  d = 0,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = NULL,
  n_ES_dist = NULL,
  iterations = 100,
  seed = NULL,
  warning = TRUE,
  average_power = TRUE
)

Arguments

`J`	Number of studies. Can be one value or a vector of multiple values.
`mu`	Effect size of practical concern. Can be one value or a vector of multiple values.
`tau`	Between-study SD. Can be one value or a vector of multiple values.
`omega`	Within-study SD. Can be one value or a vector of multiple values.
`rho`	Correlation coefficient between effect size estimates from the same study. Can be one value or a vector of multiple values.
`alpha`	Level of statistical significance. Can be one value or a vector of multiple values. Default is 0.05.
`d`	Contrast value. Can be one value or a vector of multiple values. Default is 0.
`model`	Assumed working model for dependent effect sizes, either `"CHE"` for the correlated-and-hierarchical effects model, `"CE"` for the correlated effects model, or `"MLMA"` for the multi-level meta-analysis model. Default is `"CHE"`. Can be one value or a vector of multiple values.
`var_df`	Indicates the technique used to obtain the sampling variance of the average effect size estimate and the degrees of freedom, either `"Model"` for model-based variance estimator with degrees of freedom of `J - 1`, `"Satt"` for model-based variance estimator with Satterthwaite degrees of freedom, or `"RVE"` for robust variance estimator with Satterthwaite degrees of freedom. Default is `"RVE"`. Can be one value or a vector of multiple values.
`sigma2_dist`	Distribution of sampling variance estimates from each study. Can be either a single value, a vector of plausible values, or a function that generates random values.
`n_ES_dist`	Distribution of the number of effect sizes per study. Can be either a single value, a vector of plausible values, or a function that generates random values.
`iterations`	Number of iterations per condition (default is 100).
`seed`	Numerical value for a seed to ensure reproducibility of the iterated power approximations.
`warning`	Logical indicating whether to return a warning when either sigma2_dist or n_ES_dist is based on balanced assumptions.
`average_power`	Logical indicating whether to calculate average power across the iterations for each condition.

Details

Find all background material behind the power approximations in Vembye, Pustejovsky, & Pigott (2022), including arguments for why it is suggested neither to conduct power analysis based on balanced assumptions about the number of effects per study and the study variance nor to use the original power approximation assuming independence among effect sizes (Hedges & Pigott, 2001).

Value

Returns a tibble with information about the expectation of the number of studies, the effect size of practical concern, the between-study and within-study variance components, the sample correlation, the contrast effect, the level of statistical significance, the sampling variance of overall average effect size of practical concern, the degrees of freedom, the power, the mcse, the number of iterations, the model to handle dependent effect sizes, and the methods used to obtain sampling variance estimates as well as the number effect sizes per study.

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (2022). Power approximations for overall average effects in meta-analysis with dependent effect sizes. Journal of Educational and Behavioral Statistics, 1–33. doi:10.3102/10769986221127379

Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6(3), 203–217. doi:10.1037/1082-989X.6.3.203

Examples

power <- power_MADE(
   J = c(40, 60),
   mu = 0.2,
   tau = 0.2,
   omega = 0.1,
   rho = 0.7,
   sigma2_dist = \(x) rgamma(x, shape = 5, rate = 10),
   n_ES_dist = \(x) 1 + stats::rpois(x, 5.5 - 1),
   model = c("CHE", "MLMA", "CE"),
   var_df = c("Model", "Satt", "RVE"),
   alpha = .05,
   seed = 10052510,
   iterations = 5
 )

power




power <- power_MADE(
   J = c(40, 60),
   mu = 0.2,
   tau = 0.2,
   omega = 0.1,
   rho = 0.7,
   sigma2_dist = \(x) rgamma(x, shape = 5, rate = 10),
   n_ES_dist = \(x) 1 + stats::rpois(x, 5.5 - 1),
   model = c("CHE", "MLMA", "CE"),
   var_df = c("Model", "Satt", "RVE"),
   alpha = .05,
   seed = 10052510,
   iterations = 5
 )

power

Between-Study Variance Approximation Function

Description

Rough approximation of the between-study variance based on assumption about the typical sample size of studies included in the synthesis

Usage

tau2_approximation(sample_size = 100, es, df_minus2 = TRUE)
tau2_approximation(sample_size = 100, es, df_minus2 = TRUE)

Arguments

`sample_size`	Typical sample size of studies
`es`	Smallest effect size of practical concern
`df_minus2`	If degrees of freedom should be df-2 or just df

Value

A tibble with small, medium, and large magnitudes of tau2

Examples


tau2_approximation(
sample_size = 50,
es = 0.1,
df_minus2 = TRUE
)


tau2_approximation(
sample_size = 50,
es = 0.1,
df_minus2 = TRUE
)

Co-Teaching Dataset

Description

Data from a meta-analysis on the effects of collaborative models of instruction on student achievement from Vembye, Weiss, and Bhat (2023).

Usage

VWB23_pilot
VWB23_pilot

Format

A tibble with 76 rows/studies and 9 variables

study_year: Study author and year of publication
studyid: Unique study ID
esid: Unique effect size ID
kj: Number of effect sizes per study
N_meanj: Average sample size of study
Nt_meanj: Average sample size of treatment group within study
Nc_meanj: Average sample size of control group within study
ESS_meanj: Roughly approximated effective sample sizes
vg_ms_mean: Average cluster bias corrected sampling variance estimates

Source

Find background material on Vembye's OSF page, and the preprint at https://osf.io/preprints/metaarxiv/mq5v7/.

References

Vembye, M. H., Weiss, F., & Bhat, B. H. (2023). The Effects Co-Teaching and Related Collaborative Models of Instruction on Student Achievement: A Systematic Review and Meta-Analysis. Review of Educational Research, doi:10.3102/00346543231186588

Package 'POMADE'

Help Index

Cluster Bias Correction

Description

Usage

Arguments

Value

Examples

Approximate Effective Sample Sizes

Description

Usage

Arguments

Details

Value

Examples

Minimum Detectable Effect Size (MDES) for Meta-Analysis With Dependent Effect Sizes

Description

Usage

Arguments

Value

Examples

Finding the Number of Studies Needed to Obtain a Certain Amount of Power

Description

Usage

Arguments

Value

Examples

Generic plot function for 'MADE' objects

Description

Usage

Arguments

Value

References

See Also

Examples

Plot function for a 'mdes' object

Description

Usage

Arguments

Details

Value

References

See Also

Examples

Plot function for a 'min_studies' object

Description

Usage

Arguments

Details

Value

References

See Also

Examples

Plot function for a 'power' object

Description

Usage

Arguments

Details

Value

References

See Also

Examples

Power Approximation for Overall Average Effects in Meta-Analysis With Dependent Effect Sizes

Description

Usage

Arguments

Details

Value

References

Examples

Between-Study Variance Approximation Function

Description

Usage

Arguments

Value

Examples

Co-Teaching Dataset

Description

Usage

Format