Adjust \(p\)-values for simple multiple-testing procedures

This is a modification of the mt.rawp2adjp function from the Bioconductor package multtest. We fixed an error wherein selecting the "TSBH" option overwrote the results of any previous adjustment methods, and another error created when the "BY" and "TSBH" methods were called simultaneously. We did not write the original function. For more information, see https://www.bioconductor.org/packages/3.7/bioc/manuals/multtest/man/multtest.pdf.

ControlFDR(
  rawp,
  proc = c("BH", "BY", "ABH", "TSBH", "Bonferroni", "Holm", "Hochberg", "SidakSS",
    "SidakSD"),
  alpha = 0.05,
  na.rm = FALSE,
  as.multtest.out = FALSE
)

Arguments

rawp	A vector of raw (unadjusted) \(p\)-values for each hypothesis under consideration. These could be nominal \(p\)-values, for example, from \(t\)-tables, or permutation \(p\)-values.
proc	A vector of character strings containing the names of the multiple testing procedures for which adjusted \(p\)-values are to be computed. This vector should include any of the options listed in the "Details" Section. Adjusted \(p\)-values are computed for simple FWER- and FDR- controlling procedures based on a vector of raw (unadjusted) \(p\)-values. Defaults to "BH".
alpha	A nominal Type-I error rate, or a vector of error rates, used for estimating the number of true null hypotheses in the two-stage Benjamini & Hochberg procedure ("TSBH"). Default is 0.05.
na.rm	An option for handling NA values in a list of raw \(p\)- values. If FALSE, the number of hypotheses considered is the length of the vector of raw \(p\)-values. Otherwise, if TRUE, the number of hypotheses is the number of raw \(p\)-values which were not NAs.
as.multtest.out	Should the output match the output from the mt.rawp2adjp function? If not, the output will match the input (a vector). Defaults to FALSE.

Value

A vector of the same length and order as rawp, unless the user specifies that the output should match the output from the multtest package. In that case, the use should specify as.multtest.out = TRUE and this function will return output identical to that of the mt.rawp2adjp function from package multtest. That output is as follows:

• adjp : A matrix of adjusted \(p\)-values, with rows corresponding to hypotheses and columns to multiple testing procedures. Hypotheses are sorted in increasing order of their raw (unadjusted) \(p\)-values.

• index : A vector of row indices, between 1 and length(rawp), where rows are sorted according to their raw (unadjusted) \(p\)-values. To obtain the adjusted \(p\)-values in the original data order, use adjp\[order(index),\].

• h0.ABH : The estimate of the number of true null hypotheses (as proposed by Benjamini & Hochberg (2000)) used when computing adjusted \(p\)-values for the "ABH" procedure (see Dudoit et al., 2007).

• h0.TSBH : The estimate (or vector of estimates) of the number of true null hypotheses (as proposed by Benjamini et al. (2006)) when computing adjusted \(p\)-values for the "TSBH" procedure (see Dudoit et al., 2007).

Details

This function computes adjusted \(p\)-values for simple multiple testing procedures from a vector of raw (unadjusted) \(p\)-values. The procedures include the Bonferroni, Holm (1979), Hochberg (1988), and Sidak procedures for strong control of the family-wise Type-I error rate (FWER), and the Benjamini & Hochberg (1995) and Benjamini & Yekutieli (2001) procedures for (strong) control of the false discovery rate (FDR). The less conservative adaptive Benjamini & Hochberg (2000) and two-stage Benjamini & Hochberg (2006) FDR-controlling procedures are also included.

The proc options are

• "BH" : Adjusted \(p\)-values for the Benjamini & Hochberg (1995) step-up FDR-controlling procedure (independent and positive regression dependent test statistics).

• "BY" : Adjusted \(p\)-values for the Benjamini & Yekutieli (2001) step-up FDR-controlling procedure (general dependency structures).

• "ABH" : Adjusted \(p\)-values for the adaptive Benjamini & Hochberg (2000) step-up FDR-controlling procedure. This method amends the original step-up procedure using an estimate of the number of true null hypotheses obtained from \(p\)-values. This method is not guaranteed to return finite values.

• "TSBH" : Adjusted \(p\)-values for the two-stage Benjamini & Hochberg (2006) step-up FDR-controlling procedure. This method amends the original step-up procedure using an estimate of the number of true null hypotheses obtained from a first-pass application of "BH". The adjusted \(p\)-values are \(\alpha\)- dependent, therefore \(\alpha\) must be set in the function arguments when using this procedure.

• "Bonferroni" : Bonferroni single-step adjusted \(p\)- values for strong control of the FWER.

• "Holm" : Holm (1979) step-down adjusted \(p\)-values for strong control of the FWER.

• "Hochberg" : Hochberg (1988) step-up adjusted \(p\)- values for strong control of the FWER (for raw (unadjusted) \(p\)- values satisfying the Simes inequality).

• "SidakSS" : Sidak single-step adjusted \(p\)-values for strong control of the FWER (for positive orthant dependent test statistics).

• "SidakSD" : Sidak step-down adjusted \(p\)-values for strong control of the FWER (for positive orthant dependent test statistics).

See also

AESPCA_pVals SuperPCA_pVals

Examples

  # DO NOT CALL THIS FUNCTION DIRECTLY.
  # Call this function through AESPCA_pVals() or SuperPCA_pVals() instead.

if (FALSE) {
  ###  Load the Example Data  ###
  data("colonSurv_df")
  data("colon_pathwayCollection")

  ###  Create an OmicsSurv Object  ###
  colon_Omics <- CreateOmics(
    assayData_df = colonSurv_df[, -(2:3)],
    pathwayCollection_ls = colon_pathwayCollection,
    response = colonSurv_df[, 1:3],
    respType = "surv"
  )

  ###  Extract Pathway PCs and Loadings  ###
  colonPCs_ls <- ExtractAESPCs(
    object = colon_Omics,
    parallel = TRUE,
    numCores = 2
  )

  ###  Pathway p-Values  ###
  pVals <- PermTestSurv(
    OmicsSurv = colon_Omics,
    pathwayPCs_ls = colonPCs_ls$PCs,
    parallel = TRUE,
    numCores = 2
  )

  ###  Adjust p-Values  ###
  ControlFDR(rawp = pVals)
}