t]eFdZddlZddlZddlZddlZddlZddlZddlm Z ddl m Z ddl m Z ddlmZddlmZddlmZdd lmZ dd lmZmZn#eef$rdd lmZdd lmZYnwxYwd dlmZd dlm Z d dl!m"Z"d dl#m$Z$d dl%m&Z&m'Z'm(Z(d dl)m*Z*d dl+m,Z,m-Z-d dl.m/Z/m0Z0m1Z1d dl2m3Z3ddl4m5Z5m6Z6eej7Z8e&e$j9e,e-Gddee Z:dS)z9Bagging classifier trained on balanced bootstrap samples.N)clone)BaggingClassifier)_parallel_decision_function)_partition_estimators)DecisionTreeClassifier) parse_version)check_is_fitted)Paralleldelayed)r )r )_ParamsValidationMixin)Pipeline)RandomUnderSampler)BaseUnderSampler) Substitutioncheck_sampling_strategycheck_target_type) available_if)_n_jobs_docstring_random_state_docstring) HasMethodsInterval StrOptions) _fit_context)_bagging_parameter_constraints_estimator_has)sampling_strategyn_jobs random_statec eZdZdZeedkrejej Z neje Z e e e jdddehdeegdged gd gd  dd d ddddddd d dd dd fd ZfdZefdZedZedZedfdZdfd ZeeddZfdZ xZ!S) BalancedBaggingClassifieruA Bagging classifier with additional balancing. This implementation of Bagging is similar to the scikit-learn implementation. It includes an additional step to balance the training set at fit time using a given sampler. This classifier can serves as a basis to implement various methods such as Exactly Balanced Bagging [6]_, Roughly Balanced Bagging [7]_, Over-Bagging [6]_, or SMOTE-Bagging [8]_. Read more in the :ref:`User Guide `. Parameters ---------- estimator : estimator object, default=None The base estimator to fit on random subsets of the dataset. If None, then the base estimator is a decision tree. .. versionadded:: 0.10 n_estimators : int, default=10 The number of base estimators in the ensemble. max_samples : int or float, default=1.0 The number of samples to draw from X to train each base estimator. - If int, then draw ``max_samples`` samples. - If float, then draw ``max_samples * X.shape[0]`` samples. max_features : int or float, default=1.0 The number of features to draw from X to train each base estimator. - If int, then draw ``max_features`` features. - If float, then draw ``max_features * X.shape[1]`` features. bootstrap : bool, default=True Whether samples are drawn with replacement. .. note:: Note that this bootstrap will be generated from the resampled dataset. bootstrap_features : bool, default=False Whether features are drawn with replacement. oob_score : bool, default=False Whether to use out-of-bag samples to estimate the generalization error. warm_start : bool, default=False When set to True, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble. {sampling_strategy} replacement : bool, default=False Whether or not to randomly sample with replacement or not when `sampler is None`, corresponding to a :class:`~imblearn.under_sampling.RandomUnderSampler`. {n_jobs} {random_state} verbose : int, default=0 Controls the verbosity of the building process. sampler : sampler object, default=None The sampler used to balanced the dataset before to bootstrap (if `bootstrap=True`) and `fit` a base estimator. By default, a :class:`~imblearn.under_sampling.RandomUnderSampler` is used. .. versionadded:: 0.8 base_estimator : estimator object, default=None The base estimator to fit on random subsets of the dataset. If None, then the base estimator is a decision tree. .. deprecated:: 0.10 `base_estimator` was renamed to `estimator` in version 0.10 and will be removed in 0.12. Attributes ---------- estimator_ : estimator The base estimator from which the ensemble is grown. .. versionadded:: 0.10 base_estimator_ : estimator The base estimator from which the ensemble is grown. .. deprecated:: 1.2 `base_estimator_` is deprecated in `scikit-learn` 1.2 and will be removed in 1.4. Use `estimator_` instead. When the minimum version of `scikit-learn` supported by `imbalanced-learn` will reach 1.4, this attribute will be removed. n_features_ : int The number of features when `fit` is performed. .. deprecated:: 1.0 `n_features_` is deprecated in `scikit-learn` 1.0 and will be removed in version 1.2. When the minimum version of `scikit-learn` supported by `imbalanced-learn` will reach 1.2, this attribute will be removed. estimators_ : list of estimators The collection of fitted base estimators. sampler_ : sampler object The validate sampler created from the `sampler` parameter. estimators_samples_ : list of ndarray The subset of drawn samples (i.e., the in-bag samples) for each base estimator. Each subset is defined by a boolean mask. estimators_features_ : list of ndarray The subset of drawn features for each base estimator. classes_ : ndarray of shape (n_classes,) The classes labels. n_classes_ : int or list The number of classes. oob_score_ : float Score of the training dataset obtained using an out-of-bag estimate. oob_decision_function_ : ndarray of shape (n_samples, n_classes) Decision function computed with out-of-bag estimate on the training set. If n_estimators is small it might be possible that a data point was never left out during the bootstrap. In this case, ``oob_decision_function_`` might contain NaN. n_features_in_ : int Number of features in the input dataset. .. versionadded:: 0.9 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during `fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 0.9 See Also -------- BalancedRandomForestClassifier : Random forest applying random-under sampling to balance the different bootstraps. EasyEnsembleClassifier : Ensemble of AdaBoost classifier trained on balanced bootstraps. RUSBoostClassifier : AdaBoost classifier were each bootstrap is balanced using random-under sampling at each round of boosting. Notes ----- This is possible to turn this classifier into a balanced random forest [5]_ by passing a :class:`~sklearn.tree.DecisionTreeClassifier` with `max_features='auto'` as a base estimator. See :ref:`sphx_glr_auto_examples_ensemble_plot_comparison_ensemble_classifier.py`. References ---------- .. [1] L. Breiman, "Pasting small votes for classification in large databases and on-line", Machine Learning, 36(1), 85-103, 1999. .. [2] L. Breiman, "Bagging predictors", Machine Learning, 24(2), 123-140, 1996. .. [3] T. Ho, "The random subspace method for constructing decision forests", Pattern Analysis and Machine Intelligence, 20(8), 832-844, 1998. .. [4] G. Louppe and P. Geurts, "Ensembles on Random Patches", Machine Learning and Knowledge Discovery in Databases, 346-361, 2012. .. [5] C. Chen Chao, A. Liaw, and L. Breiman. "Using random forest to learn imbalanced data." University of California, Berkeley 110, 2004. .. [6] R. Maclin, and D. Opitz. "An empirical evaluation of bagging and boosting." AAAI/IAAI 1997 (1997): 546-551. .. [7] S. Hido, H. Kashima, and Y. Takahashi. "Roughly balanced bagging for imbalanced data." Statistical Analysis and Data Mining: The ASA Data Science Journal 2.5‐6 (2009): 412-426. .. [8] S. Wang, and X. Yao. "Diversity analysis on imbalanced data sets by using ensemble models." 2009 IEEE symposium on computational intelligence and data mining. IEEE, 2009. Examples -------- >>> from collections import Counter >>> from sklearn.datasets import make_classification >>> from sklearn.model_selection import train_test_split >>> from sklearn.metrics import confusion_matrix >>> from imblearn.ensemble import BalancedBaggingClassifier >>> X, y = make_classification(n_classes=2, class_sep=2, ... weights=[0.1, 0.9], n_informative=3, n_redundant=1, flip_y=0, ... n_features=20, n_clusters_per_class=1, n_samples=1000, random_state=10) >>> print('Original dataset shape %s' % Counter(y)) Original dataset shape Counter({{1: 900, 0: 100}}) >>> X_train, X_test, y_train, y_test = train_test_split(X, y, ... random_state=0) >>> bbc = BalancedBaggingClassifier(random_state=42) >>> bbc.fit(X_train, y_train) BalancedBaggingClassifier(...) >>> y_pred = bbc.predict(X_test) >>> print(confusion_matrix(y_test, y_pred)) [[ 23 0] [ 2 225]] z1.3rrright)closed> not majority not minorityallautomajorityboolean fit_resampleN)r replacementsampler g?TFr( deprecated) max_samples max_features bootstrapbootstrap_features oob_score warm_startrr,rr verboser-base_estimatorc tjtj}d|i}d|jvr||d<n||_tjdi||||||||| | | d | |_| |_||_dS)Nr7 estimator) n_estimatorsr0r1r2r3r4r5rr r6) inspect signaturesuper__init__ parametersr9rr,r-)selfr9r:r0r1r2r3r4r5rr,rr r6r-r7bagging_classifier_signatureestimator_params __class__s :lib/python3.11/site-packages/imblearn/ensemble/_bagging.pyr?z"BalancedBaggingClassifier.__init__s((/'89I'J'J$,n= 6A A A,5 [ ) )&DN  %#%1!% "3& c@t|}tjtrTjjdkrDfdtj|jjD_ n j_ |S)Nbypasscji|]/\}}tjj|kdd|0S)r)npwhereclasses_).0keyvaluerAs rE z9BalancedBaggingClassifier._validate_y..QsJ'''C#-..q1!4e'''rF) r> _validate_y isinstancerdictsampler__sampling_typeritems_sampling_strategy)rAy y_encodedrDs` rErQz%BalancedBaggingClassifier._validate_yKsGG''** t-t 4 4 = ,88''''"9*M0##%'' '''D # #'+& %  '; ; ;W  > %"4>22NN  (< < < M+    #4#677NN"7^^N = '8 3 3 M $ $t7N $ O O O" ',)G H   #'?D     DD s C!! C/.C/c|jS)z$Estimator used to grow the ensemble.)rarAs rE estimator_z$BalancedBaggingClassifier.estimator_s rFcDtjdt|jS)z-Number of features when ``fit`` is performed.z`n_features_` was deprecated in scikit-learn 1.0. This attribute will not be accessible when the minimum supported version of scikit-learn is 1.2.)r]r^r_n_features_in_rgs rE n_features_z%BalancedBaggingClassifier.n_features_s-        ""rF)prefer_skip_nested_validationcp|t||S)a+Build a Bagging ensemble of estimators from the training set (X, y). Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The training input samples. Sparse matrices are accepted only if they are supported by the base estimator. y : array-like of shape (n_samples,) The target values (class labels in classification, real numbers in regression). Returns ------- self : object Fitted estimator. )_validate_paramsr>fit)rAXrXrDs rErozBalancedBaggingClassifier.fits/( ww{{1a   rFct||jt|j|_nt |j|_t |||jdS)N)r,) sample_weight) rr-rr,rTrr>_fitr0)rArprXr0 max_depthrrrDs rErszBalancedBaggingClassifier._fitso! < . ,DMM"$,//DMww||Aq$"2$|GGGrFdecision_functionc@tddgdddtjj\}}t |jfdt|D}t|jz }|S)aAverage of the decision functions of the base classifiers. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The training input samples. Sparse matrices are accepted only if they are supported by the base estimator. Returns ------- score : ndarray of shape (n_samples, k) The decision function of the input samples. The columns correspond to the classes in sorted order, as they appear in the attribute ``classes_``. Regression and binary classification are special cases with ``k == 1``, otherwise ``k==n_classes``. csrcscNF) accept_sparsedtypeforce_all_finitereset)rr6c 3K|][}ttj||dzj||dzV\dS)rN)r r estimators_estimators_features_)rMirprAstartss rE z>BalancedBaggingClassifier.decision_function..sF F  1G/ 0 0 VAE]!:;)&)fQUm*CD  F F F F F F rF) r _validate_datarr:rr r6rangesum)rArpr_ all_decisions decisionsrs`` @rEruz+BalancedBaggingClassifier.decision_functions$      %."   2$2CT[QQ6E EEEF F F F F F 6]] F F F    &&):: rFct}d}d}d}||vr ||||<n||i||<|S)N _xfail_checkscheck_estimators_nan_infz9Fails because the sampler removed infinity and NaN values)r> _more_tags)rAtagstags_key failing_testreasonrDs rErz$BalancedBaggingClassifier._more_tagssVww!!##"1 L t  +1DN< ( (*F3DN rF)Nr.)NNN)"__name__ __module__ __qualname____doc__sklearn_versionrcopydeepcopyr_parameter_constraintsrupdaternumbersRealrrScallablerr?rQrrepropertyrhrkrrorsrrrur __classcell__)rDs@rEr"r"+sU YYx--....!./@/W!X!X!./M!N!N!!q!G<<< VVVWW " &;" N#344d;    *   ##*******X$+A*@*B*B    FX ##X#\666!!!!76!, H H H H H H \..!45566++76+Z         rFr");rrr<rr]numpyrJsklearn sklearn.basersklearn.ensemblersklearn.ensemble._baggingrsklearn.ensemble._baser sklearn.treer sklearn.utilsrsklearn.utils.validationr sklearn.utils.parallelr r ImportErrorModuleNotFoundErrorjoblibsklearn.utils.fixesbaser pipelinerunder_samplingrunder_sampling.baserutilsrrrutils._available_ifrutils._docstringrrutils._param_validationrrr utils.fixesr_commonrr __version__r_sampling_strategy_docstringr"r;rFrErs??  ......AAAAAA888888//////''''''444444,888888888(),,,++++++++,*)))))//////222222LLLLLLLLLL......IIIIIIIIFFFFFFFFFF&&&&&&CCCCCCCC- 344&C (  68I  sAA%$A%