This function run tests, that help to evaluate if data in folds is (a) stratified and (b) blocked.
cvo_test_bs( obj, stratify_by = NULL, block_by = NULL, data = NULL, n_col_show = 10 )
| obj | A list with validation/test set indices in folds. Note: If indices are from training set, the result will be incorrect. |
|---|---|
| stratify_by | A name of variable used for stratification. |
| block_by | A name of variable used for blocking. |
| data | A data frame, for which |
| n_col_show | Number of blocking variable cross-tabulation's columns to be shown. Default is 10. |
Print tables that help to evaluate if data is (a) stratified, (b) blocked.
Vilmantas Gegzna
library(manyROC) # [!!!] Load data DataSet1 <- data.frame(ID = rep(1:20, each = 2), gr = gl(4, 10, labels = LETTERS[1:4]), .row = 1:40) obj <- cvo_create_folds(data = DataSet1, stratify_by = "gr", block_by = "ID", returnTrain = FALSE) cvo_test_bs(obj, stratify_by = "gr", block_by = "ID", data = DataSet1)#> ************************************************************\n____________________________________________________________\n Test for STRATIFICATION #> #> A B C D <<< >>> A B C D #> Rep1_Fold1 2 2 2 2 <-Counts | Proportions-> 0.25 0.25 0.25 0.25 #> Rep1_Fold2 2 2 2 2 <-Counts | Proportions-> 0.25 0.25 0.25 0.25 #> Rep1_Fold3 2 2 2 2 <-Counts | Proportions-> 0.25 0.25 0.25 0.25 #> Rep1_Fold4 2 2 2 2 <-Counts | Proportions-> 0.25 0.25 0.25 0.25 #> Rep1_Fold5 2 2 2 2 <-Counts | Proportions-> 0.25 0.25 0.25 0.25 #> #> If stratified, the proportions of each group in each fold #> (row) should be (approximately) equal and with no zero values. #> Test is not valid if data is blocked and number of cases in #> each block differs significantly. #> ____________________________________________________________\n Test for BLOCKING: BLOCKED #> #> ID #> 1 2 3 4 5 6 7 8 9 10 .. #> Rep1_Fold1 0 0 0 0 2 2 0 0 0 0 .. #> Rep1_Fold2 0 0 0 2 0 0 0 0 2 0 .. #> Rep1_Fold3 0 0 2 0 0 0 0 0 0 2 .. #> Rep1_Fold4 0 2 0 0 0 0 0 2 0 0 .. #> Rep1_Fold5 2 0 0 0 0 0 2 0 0 0 .. #> #> Table shows number of observations in each fold. #> If blocked, the same ID appears just in one fold. #> 10 (of 20) first columns are displayed. #> ************************************************************\n# > ************************************************************ # > Test for STRATIFICATION # > # > A B C D <<< >>> A B C D # > Fold1 2 2 2 2 <-Counts | Proportions-> 0.25 0.25 0.25 0.25 # > Fold2 2 2 2 2 <-Counts | Proportions-> 0.25 0.25 0.25 0.25 # > Fold3 2 2 2 2 <-Counts | Proportions-> 0.25 0.25 0.25 0.25 # > Fold4 2 2 2 2 <-Counts | Proportions-> 0.25 0.25 0.25 0.25 # > Fold5 2 2 2 2 <-Counts | Proportions-> 0.25 0.25 0.25 0.25 # > # > If stratified, the proportions of each group in each fold # > (row) should be (approximately) equal and with no zero values. # > ____________________________________________________________ # > Test for BLOCKING: BLOCKED # > # > 1 2 3 4 5 6 7 8 9 10 .. # > Fold1 0 0 0 0 2 0 0 2 0 0 .. # > Fold2 2 0 0 0 0 2 0 0 0 0 .. # > Fold3 0 0 2 0 0 0 0 0 2 0 .. # > Fold4 0 0 0 2 0 0 2 0 0 0 .. # > Fold5 0 2 0 0 0 0 0 0 0 2 .. # > # > Number of unique IDs in each fold (first 10 columns). # > If blocked, the same ID appears just in one fold. # > ************************************************************