Sequentially Bootstrapped

In sampling section we have shown that sampling should be done by Sequential Bootstrapping. SequentiallyBootstrappedBaggingClassifier and SequentiallyBootstrappedBaggingRegressor classes extend sklearn’s BaggingClassifier and BaggingRegressor classes by using Sequential Bootstrapping instead of random sampling.

In order to build indicator matrix we need Triple Barrier Events (samples_info_sets) and price bars used to label training data set. That is why samples_info_sets and price_bars are input parameters for classifier/regressor.

To better understand the underlying method, you may be interested in reading the Sampling/Sequential Bootstrapping.

Implementation

Implementation of Sequentially Bootstrapped Bagging Classifier using sklearn’s library as base class.

class mlfinpy.ensemble.sb_bagging.SequentiallyBootstrappedBaggingClassifier(samples_info_sets, price_bars, estimator=None, n_estimators=10, max_samples=1.0, max_features=1.0, bootstrap_features=False, oob_score=False, warm_start=False, n_jobs=None, random_state=None, verbose=0)[source]

A Sequentially Bootstrapped Bagging classifier is an ensemble meta-estimator that fits base classifiers each on random subsets of the original dataset generated using Sequential Bootstrapping sampling procedure and then aggregate their individual predictions (either by voting or by averaging) to form a final prediction. Such a meta-estimator can typically be used as a way to reduce the variance of a black-box estimator (e.g., a decision tree), by introducing randomization into its construction procedure and then making an ensemble out of it.

Parameters

samples_info_setspd.Series,

The information range on which each record is constructed from samples_info_sets.index: Time when the information extraction started. samples_info_sets.value: Time when the information extraction ended.

price_barspd.DataFrame,

Price bars used in samples_info_sets generation

estimatorobject or None, optional (default=None)

The base estimator to fit on random subsets of the dataset. If None, then the base estimator is a decision tree.

n_estimatorsint, optional (default=10)

The number of base estimators in the ensemble.

max_samplesint or float, optional (default=1.0)

The number of samples to draw from X to train each base estimator. If int, then draw max_samples samples. If float, then draw max_samples * X.shape[0] samples.

max_featuresint or float, optional (default=1.0)

The number of features to draw from X to train each base estimator. If int, then draw max_features features. If float, then draw max_features * X.shape[1] features.

bootstrap_featuresbool, optional (default=False)

Whether features are drawn with replacement.

oob_scorebool, optional (default=False)

Whether to use out-of-bag samples to estimate the generalization error.

warm_startbool, optional (default=False)

When set to True, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble.

n_jobsint or None, optional (default=None)

The number of jobs to run in parallel for both fit and predict. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors.

random_stateint, RandomState instance or None, optional (default=None)

If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by np.random.

verboseint, optional (default=0)

Controls the verbosity when fitting and predicting.

Attributes

estimator_estimator

The base estimator from which the ensemble is grown.

estimators_list of estimators

The collection of fitted base estimators.

estimators_samples_list of arrays

The subset of drawn samples (i.e., the in-bag samples) for each base estimator. Each subset is defined by an array of the indices selected.

estimators_features_list of arrays

The subset of drawn features for each base estimator.

classes_array of shape = [n_classes]

The classes labels.

n_classes_int or list

The number of classes.

oob_score_float

Score of the training dataset obtained using an out-of-bag estimate.

oob_decision_function_array of shape = [n_samples, n_classes]

Decision function computed with out-of-bag estimate on the training set. If n_estimators is small it might be possible that a data point was never left out during the bootstrap. In this case, oob_decision_function_ might contain NaN.

set_fit_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → SequentiallyBootstrappedBaggingClassifier

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in fit.

Returns

selfobject

The updated object.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → SequentiallyBootstrappedBaggingClassifier

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns

selfobject

The updated object.

class mlfinpy.ensemble.sb_bagging.SequentiallyBootstrappedBaggingRegressor(samples_info_sets, price_bars, estimator=None, n_estimators=10, max_samples=1.0, max_features=1.0, bootstrap_features=False, oob_score=False, warm_start=False, n_jobs=None, random_state=None, verbose=0)[source]

A Sequentially Bootstrapped Bagging regressor is an ensemble meta-estimator that fits base regressors each on random subsets of the original dataset and then aggregates their individual predictions to form a final prediction.

Parameters

samples_info_setspd.Series

The information range on which each record is constructed from samples_info_sets.index: Time when the information extraction started. samples_info_sets.value: Time when the information extraction ended.

price_barspd.DataFrame

Price bars used in samples_info_sets generation

estimatorobject or None, optional (default=None)

The base estimator to fit on random subsets of the dataset. If None, then the base estimator is a decision tree.

n_estimatorsint, optional (default=10)

The number of base estimators in the ensemble.

max_samplesint or float, optional (default=1.0)

The number of samples to draw from X to train each base estimator. If int, then draw max_samples samples. If float, then draw max_samples * X.shape[0] samples.

max_featuresint or float, optional (default=1.0)

The number of features to draw from X to train each base estimator. If int, then draw max_features features. If float, then draw max_features * X.shape[1] features.

bootstrap_featuresbool, optional (default=False)

Whether features are drawn with replacement.

oob_scorebool

Whether to use out-of-bag samples to estimate the generalization error.

warm_startbool, optional (default=False)

When set to True, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble.

n_jobsint or None, optional (default=None)

The number of jobs to run in parallel for both fit and predict. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors.

random_stateint, RandomState instance or None, optional (default=None)

If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by np.random.

verboseint, optional (default=0)

Controls the verbosity when fitting and predicting.

Attributes

estimators_list of estimators

The collection of fitted sub-estimators.

estimators_samples_list of arrays

The subset of drawn samples (i.e., the in-bag samples) for each base estimator. Each subset is defined by an array of the indices selected.

estimators_features_list of arrays

The subset of drawn features for each base estimator.

oob_score_float

Score of the training dataset obtained using an out-of-bag estimate.

oob_prediction_array of shape = [n_samples]

Prediction computed with out-of-bag estimate on the training set. If n_estimators is small it might be possible that a data point was never left out during the bootstrap. In this case, oob_prediction_ might contain NaN.

set_fit_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → SequentiallyBootstrappedBaggingRegressor

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in fit.

Returns

selfobject

The updated object.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → SequentiallyBootstrappedBaggingRegressor

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns

selfobject

The updated object.

Example

An example of using SequentiallyBootstrappedBaggingClassifier.

import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from mlfinlab.ensemble import SequentiallyBootstrappedBaggingClassifier

X = pd.read_csv('X_FILE_PATH', index_col=0, parse_dates = [0])
y = pd.read_csv('y_FILE_PATH', index_col=0, parse_dates = [0])
triple_barrier_events = pd.read_csv('BARRIER_FILE_PATH', index_col=0, parse_dates = [0, 2])
price_bars = pd.read_csv('PRICE_BARS_FILE_PATH', index_col=0, parse_dates = [0, 2])

triple_barrier_events = triple_barrier_events.loc[X.index, :] # take only train part
triple_barrier_events = triple_barrier_events[(triple_barrier_events.index >= X.index.min()) &
                                              (triple_barrier_events.index <= X.index.max())]

base_est = RandomForestClassifier(n_estimators=1, criterion='entropy', bootstrap=False,
                                  class_weight='balanced_subsample')
clf = SequentiallyBootstrappedBaggingClassifier(base_estimator=base_est,
                                                samples_info_sets=triple_barrier_events.t1,
                                                price_bars=price_bars, oob_score=True)
clf.fit(X, y)