resample.instance

Description

Base class for specific resampling draws like cross-validation or bootstrapping. This class encapsulates training and test sets generated from the data set for a number of iterations. It mainly stores a set of integer vectors indicating the training examples for each iteration. Don't create objects from this class directly but use the corresponding subclasses. For construction simply use the factory methods of the subclasses - e.g. for cross-validation use make.cv.instance to get a cv.instance.

Slots

desc:

(resample.desc) Description object for resampling strategy

size:

(integer) Number of observations in data

inds:

(list) List of integer vectors specifying the training cases for each iteration. Each vector might contain duplicated indices and the order matters for some classifiers.

Methods

benchmark

signature(learn.task = "learn.task", outer.resampling = "resample.instance", inner.resampling = "resample.desc", ranges = "list", measure = "list", all.tune.results = "logical"): benchmark conducts a benchmark experiment for a single classifier on a single data set. This consists of an inner stage and outer stage. At the outer stage a tuning set and a test set are repeatedly formed from the data through resampling (usually cross-validation or bootstrapping). The respective hyperparameters of the classifier are tuned on the tuning set again through an inner resampling process, the classifier is trained on the complete tuning set with the best found hyperparameters and the performance is measured on the test set.

resample.fit

signature(learn.task = "learn.task", resample.instance = "resample.instance", parset = "list", vars = "character", models = "logical", type = "character"): Given the training and test indices (e.g. generated by cross-validation and generally specified by the resample.instance object) resample.fit fits the selected learner using the training sets and performs predictions for the test sets. These predictions are returned - encapsulated in a resample.result object. Optionally the fitted models are also stored.

initialize

signature( = "resample.instance"): This is mainly for internal use, you only need to use this, when you extend resample.instance to add another resampling strategy!

[

signature( = "resample.instance"): Getter.

resample.performance

signature(learn.task = "learn.task", resample.instance = "resample.instance", resample.result = "resample.result", measure = "list"): Measures the quality of predictions w.r.t. some loss function for a resampled fit.

Note

If you want to add another resampling strategy, have a look at the web documentation.

See Also

resample.desc, make.cv.instance, make.bs.instance, make.subsample.instance, resample.fit