RandomizedSearchCV를 사용하여 XGBoost 최적 하이퍼 파라미터 구하는 예제코드

RandomizedSearchCV를 사용하여 XGBoost의  최적 하이퍼 파라미터 구하는 예제코드입니다.

최초작성 2024. 5. 30

 

import pandas as pd import numpy as np from sklearn.model_selection import train_test_split, RandomizedSearchCV from sklearn.metrics import accuracy_score from xgboost import XGBClassifier from sklearn.datasets import load_iris RANDOM_SEED=42 # Iris 데이터셋 로드 iris = load_iris() df = pd.DataFrame(data=iris.data, columns=iris.feature_names) df['label'] = iris.target X = df.drop(columns=['label']) y = df['label'] # train 데이터세트와 test 데이터세트로 분리 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=RANDOM_SEED) params = {         'learning_rate': [0.001, 0.005, 0.01, 0.05, 0.1],         'min_child_weight': np.arange(1,20,1),         'gamma': np.arange(0.1, 2.1, 0.1),         'subsample': np.arange(0.1, 1.1, 0.1),         'colsample_bytree': np.arange(0.1, 1.1, 0.1),         'colsample_bylevel': np.arange(0.1, 1.1, 0.1),         'max_depth': np.arange(1, 20, 1),         'n_estimators': np.arange(500, 1000, 50),         'reg_lambda' : np.arange(0.1, 2.1, 0.1),         "reg_alpha": np.arange(0.1, 2.1, 0.1),         } xgb = XGBClassifier(objective="multi:softprob", eval_metric='mlogloss') search = RandomizedSearchCV(xgb, param_distributions=params, n_iter=20, scoring='accuracy', n_jobs=4,  verbose=3, random_state=RANDOM_SEED ) search.fit(X_train, y_train) # 최적의 하이퍼파라미터로 재학습 best_params = search.best_params_ final_model = XGBClassifier(objective="multi:softprob", eval_metric='mlogloss', **best_params) final_model.fit(X_train, y_train) y_pred = final_model.predict(X_test) accuracy = accuracy_score(y_test, y_pred) print(f'Accuracy = {accuracy:.2f}')

실행결과입니다.

 

(opencv) webnautes@webnautesui-MacBookAir Python_Example % /Users/webnautes/miniforge3/envs/opencv/bin/python /Users/webnautes/Python_Example/test.py

Fitting 5 folds for each of 20 candidates, totalling 100 fits

[CV 3/5] END colsample_bylevel=0.9, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.01, max_depth=15, min_child_weight=6, n_estimators=600, reg_alpha=1.4000000000000001, reg_lambda=1.1, subsample=0.30000000000000004;, score=0.750 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.9, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.01, max_depth=15, min_child_weight=6, n_estimators=600, reg_alpha=1.4000000000000001, reg_lambda=1.1, subsample=0.30000000000000004;, score=0.900 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.9, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.01, max_depth=15, min_child_weight=6, n_estimators=600, reg_alpha=1.4000000000000001, reg_lambda=1.1, subsample=0.30000000000000004;, score=0.800 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.9, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.01, max_depth=15, min_child_weight=6, n_estimators=600, reg_alpha=1.4000000000000001, reg_lambda=1.1, subsample=0.30000000000000004;, score=0.950 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.6, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.05, max_depth=11, min_child_weight=18, n_estimators=500, reg_alpha=0.2, reg_lambda=1.5000000000000002, subsample=0.30000000000000004;, score=0.300 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.9, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.01, max_depth=15, min_child_weight=6, n_estimators=600, reg_alpha=1.4000000000000001, reg_lambda=1.1, subsample=0.30000000000000004;, score=0.800 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.6, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.05, max_depth=11, min_child_weight=18, n_estimators=500, reg_alpha=0.2, reg_lambda=1.5000000000000002, subsample=0.30000000000000004;, score=0.300 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.6, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.05, max_depth=11, min_child_weight=18, n_estimators=500, reg_alpha=0.2, reg_lambda=1.5000000000000002, subsample=0.30000000000000004;, score=0.300 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.6, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.05, max_depth=11, min_child_weight=18, n_estimators=500, reg_alpha=0.2, reg_lambda=1.5000000000000002, subsample=0.30000000000000004;, score=0.300 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.6, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.05, max_depth=11, min_child_weight=18, n_estimators=500, reg_alpha=0.2, reg_lambda=1.5000000000000002, subsample=0.30000000000000004;, score=0.350 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.4, colsample_bytree=0.2, gamma=0.30000000000000004, learning_rate=0.001, max_depth=19, min_child_weight=2, n_estimators=600, reg_alpha=2.0, reg_lambda=1.2000000000000002, subsample=0.5;, score=1.000 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.4, colsample_bytree=0.2, gamma=0.30000000000000004, learning_rate=0.001, max_depth=19, min_child_weight=2, n_estimators=600, reg_alpha=2.0, reg_lambda=1.2000000000000002, subsample=0.5;, score=0.800 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.4, colsample_bytree=0.2, gamma=0.30000000000000004, learning_rate=0.001, max_depth=19, min_child_weight=2, n_estimators=600, reg_alpha=2.0, reg_lambda=1.2000000000000002, subsample=0.5;, score=0.950 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.4, colsample_bytree=0.2, gamma=0.30000000000000004, learning_rate=0.001, max_depth=19, min_child_weight=2, n_estimators=600, reg_alpha=2.0, reg_lambda=1.2000000000000002, subsample=0.5;, score=1.000 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.4, colsample_bytree=0.2, gamma=0.30000000000000004, learning_rate=0.001, max_depth=19, min_child_weight=2, n_estimators=600, reg_alpha=2.0, reg_lambda=1.2000000000000002, subsample=0.5;, score=0.900 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=1.0, gamma=0.9, learning_rate=0.001, max_depth=6, min_child_weight=3, n_estimators=900, reg_alpha=1.7000000000000002, reg_lambda=1.9000000000000001, subsample=0.1;, score=0.300 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=1.0, gamma=0.9, learning_rate=0.001, max_depth=6, min_child_weight=3, n_estimators=900, reg_alpha=1.7000000000000002, reg_lambda=1.9000000000000001, subsample=0.1;, score=0.450 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=1.0, gamma=0.9, learning_rate=0.001, max_depth=6, min_child_weight=3, n_estimators=900, reg_alpha=1.7000000000000002, reg_lambda=1.9000000000000001, subsample=0.1;, score=0.600 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=1.0, gamma=0.9, learning_rate=0.001, max_depth=6, min_child_weight=3, n_estimators=900, reg_alpha=1.7000000000000002, reg_lambda=1.9000000000000001, subsample=0.1;, score=0.650 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=1.0, gamma=0.9, learning_rate=0.001, max_depth=6, min_child_weight=3, n_estimators=900, reg_alpha=1.7000000000000002, reg_lambda=1.9000000000000001, subsample=0.1;, score=0.550 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.8, colsample_bytree=0.1, gamma=0.7000000000000001, learning_rate=0.1, max_depth=12, min_child_weight=12, n_estimators=900, reg_alpha=0.30000000000000004, reg_lambda=1.2000000000000002, subsample=0.5;, score=0.350 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.8, colsample_bytree=0.1, gamma=0.7000000000000001, learning_rate=0.1, max_depth=12, min_child_weight=12, n_estimators=900, reg_alpha=0.30000000000000004, reg_lambda=1.2000000000000002, subsample=0.5;, score=0.350 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.8, colsample_bytree=0.1, gamma=0.7000000000000001, learning_rate=0.1, max_depth=12, min_child_weight=12, n_estimators=900, reg_alpha=0.30000000000000004, reg_lambda=1.2000000000000002, subsample=0.5;, score=0.350 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.8, colsample_bytree=0.1, gamma=0.7000000000000001, learning_rate=0.1, max_depth=12, min_child_weight=12, n_estimators=900, reg_alpha=0.30000000000000004, reg_lambda=1.2000000000000002, subsample=0.5;, score=0.350 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.8, colsample_bytree=0.1, gamma=0.7000000000000001, learning_rate=0.1, max_depth=12, min_child_weight=12, n_estimators=900, reg_alpha=0.30000000000000004, reg_lambda=1.2000000000000002, subsample=0.5;, score=0.550 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.30000000000000004, gamma=1.3000000000000003, learning_rate=0.01, max_depth=18, min_child_weight=10, n_estimators=950, reg_alpha=1.6, reg_lambda=1.0, subsample=0.30000000000000004;, score=0.300 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.30000000000000004, gamma=1.3000000000000003, learning_rate=0.01, max_depth=18, min_child_weight=10, n_estimators=950, reg_alpha=1.6, reg_lambda=1.0, subsample=0.30000000000000004;, score=0.350 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.30000000000000004, gamma=1.3000000000000003, learning_rate=0.01, max_depth=18, min_child_weight=10, n_estimators=950, reg_alpha=1.6, reg_lambda=1.0, subsample=0.30000000000000004;, score=0.350 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.30000000000000004, gamma=1.3000000000000003, learning_rate=0.01, max_depth=18, min_child_weight=10, n_estimators=950, reg_alpha=1.6, reg_lambda=1.0, subsample=0.30000000000000004;, score=0.350 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.30000000000000004, gamma=1.3000000000000003, learning_rate=0.01, max_depth=18, min_child_weight=10, n_estimators=950, reg_alpha=1.6, reg_lambda=1.0, subsample=0.30000000000000004;, score=0.300 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.1, colsample_bytree=0.6, gamma=0.30000000000000004, learning_rate=0.01, max_depth=4, min_child_weight=10, n_estimators=600, reg_alpha=0.30000000000000004, reg_lambda=0.1, subsample=0.8;, score=1.000 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.1, colsample_bytree=0.6, gamma=0.30000000000000004, learning_rate=0.01, max_depth=4, min_child_weight=10, n_estimators=600, reg_alpha=0.30000000000000004, reg_lambda=0.1, subsample=0.8;, score=0.800 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.1, colsample_bytree=0.6, gamma=0.30000000000000004, learning_rate=0.01, max_depth=4, min_child_weight=10, n_estimators=600, reg_alpha=0.30000000000000004, reg_lambda=0.1, subsample=0.8;, score=0.900 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.1, colsample_bytree=0.6, gamma=0.30000000000000004, learning_rate=0.01, max_depth=4, min_child_weight=10, n_estimators=600, reg_alpha=0.30000000000000004, reg_lambda=0.1, subsample=0.8;, score=1.000 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.1, colsample_bytree=0.6, gamma=0.30000000000000004, learning_rate=0.01, max_depth=4, min_child_weight=10, n_estimators=600, reg_alpha=0.30000000000000004, reg_lambda=0.1, subsample=0.8;, score=0.900 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.2, gamma=0.7000000000000001, learning_rate=0.01, max_depth=8, min_child_weight=1, n_estimators=600, reg_alpha=0.5, reg_lambda=0.1, subsample=0.1;, score=1.000 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.2, gamma=0.7000000000000001, learning_rate=0.01, max_depth=8, min_child_weight=1, n_estimators=600, reg_alpha=0.5, reg_lambda=0.1, subsample=0.1;, score=0.800 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.2, gamma=0.7000000000000001, learning_rate=0.01, max_depth=8, min_child_weight=1, n_estimators=600, reg_alpha=0.5, reg_lambda=0.1, subsample=0.1;, score=0.800 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.2, gamma=0.7000000000000001, learning_rate=0.01, max_depth=8, min_child_weight=1, n_estimators=600, reg_alpha=0.5, reg_lambda=0.1, subsample=0.1;, score=1.000 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.2, gamma=0.7000000000000001, learning_rate=0.01, max_depth=8, min_child_weight=1, n_estimators=600, reg_alpha=0.5, reg_lambda=0.1, subsample=0.1;, score=0.900 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.4, colsample_bytree=0.5, gamma=1.1, learning_rate=0.05, max_depth=14, min_child_weight=12, n_estimators=850, reg_alpha=1.3000000000000003, reg_lambda=1.9000000000000001, subsample=0.6;, score=0.950 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.4, colsample_bytree=0.5, gamma=1.1, learning_rate=0.05, max_depth=14, min_child_weight=12, n_estimators=850, reg_alpha=1.3000000000000003, reg_lambda=1.9000000000000001, subsample=0.6;, score=0.850 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.4, colsample_bytree=0.5, gamma=1.1, learning_rate=0.05, max_depth=14, min_child_weight=12, n_estimators=850, reg_alpha=1.3000000000000003, reg_lambda=1.9000000000000001, subsample=0.6;, score=0.700 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.4, colsample_bytree=0.5, gamma=1.1, learning_rate=0.05, max_depth=14, min_child_weight=12, n_estimators=850, reg_alpha=1.3000000000000003, reg_lambda=1.9000000000000001, subsample=0.6;, score=0.900 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.30000000000000004, gamma=1.0, learning_rate=0.05, max_depth=5, min_child_weight=13, n_estimators=750, reg_alpha=1.4000000000000001, reg_lambda=1.9000000000000001, subsample=0.1;, score=0.300 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.4, colsample_bytree=0.5, gamma=1.1, learning_rate=0.05, max_depth=14, min_child_weight=12, n_estimators=850, reg_alpha=1.3000000000000003, reg_lambda=1.9000000000000001, subsample=0.6;, score=0.750 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.30000000000000004, gamma=1.0, learning_rate=0.05, max_depth=5, min_child_weight=13, n_estimators=750, reg_alpha=1.4000000000000001, reg_lambda=1.9000000000000001, subsample=0.1;, score=0.300 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.30000000000000004, gamma=1.0, learning_rate=0.05, max_depth=5, min_child_weight=13, n_estimators=750, reg_alpha=1.4000000000000001, reg_lambda=1.9000000000000001, subsample=0.1;, score=0.300 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.30000000000000004, gamma=1.0, learning_rate=0.05, max_depth=5, min_child_weight=13, n_estimators=750, reg_alpha=1.4000000000000001, reg_lambda=1.9000000000000001, subsample=0.1;, score=0.300 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.30000000000000004, gamma=1.0, learning_rate=0.05, max_depth=5, min_child_weight=13, n_estimators=750, reg_alpha=1.4000000000000001, reg_lambda=1.9000000000000001, subsample=0.1;, score=0.350 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.9, colsample_bytree=0.4, gamma=0.5, learning_rate=0.001, max_depth=8, min_child_weight=7, n_estimators=550, reg_alpha=0.6, reg_lambda=1.2000000000000002, subsample=0.6;, score=1.000 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.9, colsample_bytree=0.4, gamma=0.5, learning_rate=0.001, max_depth=8, min_child_weight=7, n_estimators=550, reg_alpha=0.6, reg_lambda=1.2000000000000002, subsample=0.6;, score=0.800 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.9, colsample_bytree=0.4, gamma=0.5, learning_rate=0.001, max_depth=8, min_child_weight=7, n_estimators=550, reg_alpha=0.6, reg_lambda=1.2000000000000002, subsample=0.6;, score=0.900 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.9, colsample_bytree=0.4, gamma=0.5, learning_rate=0.001, max_depth=8, min_child_weight=7, n_estimators=550, reg_alpha=0.6, reg_lambda=1.2000000000000002, subsample=0.6;, score=1.000 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.9, colsample_bytree=0.4, gamma=0.5, learning_rate=0.001, max_depth=8, min_child_weight=7, n_estimators=550, reg_alpha=0.6, reg_lambda=1.2000000000000002, subsample=0.6;, score=0.900 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.5, colsample_bytree=0.6, gamma=0.7000000000000001, learning_rate=0.001, max_depth=15, min_child_weight=11, n_estimators=650, reg_alpha=1.4000000000000001, reg_lambda=1.4000000000000001, subsample=0.5;, score=0.350 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.5, colsample_bytree=0.6, gamma=0.7000000000000001, learning_rate=0.001, max_depth=15, min_child_weight=11, n_estimators=650, reg_alpha=1.4000000000000001, reg_lambda=1.4000000000000001, subsample=0.5;, score=0.350 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.5, colsample_bytree=0.6, gamma=0.7000000000000001, learning_rate=0.001, max_depth=15, min_child_weight=11, n_estimators=650, reg_alpha=1.4000000000000001, reg_lambda=1.4000000000000001, subsample=0.5;, score=0.350 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.5, colsample_bytree=0.6, gamma=0.7000000000000001, learning_rate=0.001, max_depth=15, min_child_weight=11, n_estimators=650, reg_alpha=1.4000000000000001, reg_lambda=1.4000000000000001, subsample=0.5;, score=0.350 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.5, colsample_bytree=0.6, gamma=0.7000000000000001, learning_rate=0.001, max_depth=15, min_child_weight=11, n_estimators=650, reg_alpha=1.4000000000000001, reg_lambda=1.4000000000000001, subsample=0.5;, score=0.400 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.1, colsample_bytree=0.9, gamma=1.1, learning_rate=0.001, max_depth=13, min_child_weight=6, n_estimators=850, reg_alpha=1.3000000000000003, reg_lambda=1.1, subsample=0.1;, score=0.300 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.1, colsample_bytree=0.9, gamma=1.1, learning_rate=0.001, max_depth=13, min_child_weight=6, n_estimators=850, reg_alpha=1.3000000000000003, reg_lambda=1.1, subsample=0.1;, score=0.350 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.1, colsample_bytree=0.9, gamma=1.1, learning_rate=0.001, max_depth=13, min_child_weight=6, n_estimators=850, reg_alpha=1.3000000000000003, reg_lambda=1.1, subsample=0.1;, score=0.350 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.1, colsample_bytree=0.9, gamma=1.1, learning_rate=0.001, max_depth=13, min_child_weight=6, n_estimators=850, reg_alpha=1.3000000000000003, reg_lambda=1.1, subsample=0.1;, score=0.350 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.1, gamma=1.7000000000000002, learning_rate=0.01, max_depth=14, min_child_weight=7, n_estimators=800, reg_alpha=0.8, reg_lambda=0.9, subsample=0.1;, score=0.300 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.1, colsample_bytree=0.9, gamma=1.1, learning_rate=0.001, max_depth=13, min_child_weight=6, n_estimators=850, reg_alpha=1.3000000000000003, reg_lambda=1.1, subsample=0.1;, score=0.300 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.1, gamma=1.7000000000000002, learning_rate=0.01, max_depth=14, min_child_weight=7, n_estimators=800, reg_alpha=0.8, reg_lambda=0.9, subsample=0.1;, score=0.350 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.1, gamma=1.7000000000000002, learning_rate=0.01, max_depth=14, min_child_weight=7, n_estimators=800, reg_alpha=0.8, reg_lambda=0.9, subsample=0.1;, score=0.350 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.1, gamma=1.7000000000000002, learning_rate=0.01, max_depth=14, min_child_weight=7, n_estimators=800, reg_alpha=0.8, reg_lambda=0.9, subsample=0.1;, score=0.350 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.7000000000000001, colsample_bytree=0.1, gamma=1.7000000000000002, learning_rate=0.01, max_depth=14, min_child_weight=7, n_estimators=800, reg_alpha=0.8, reg_lambda=0.9, subsample=0.1;, score=0.350 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.2, colsample_bytree=0.2, gamma=1.5000000000000002, learning_rate=0.01, max_depth=11, min_child_weight=9, n_estimators=800, reg_alpha=1.9000000000000001, reg_lambda=0.1, subsample=0.9;, score=1.000 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.2, colsample_bytree=0.2, gamma=1.5000000000000002, learning_rate=0.01, max_depth=11, min_child_weight=9, n_estimators=800, reg_alpha=1.9000000000000001, reg_lambda=0.1, subsample=0.9;, score=0.800 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.2, colsample_bytree=0.2, gamma=1.5000000000000002, learning_rate=0.01, max_depth=11, min_child_weight=9, n_estimators=800, reg_alpha=1.9000000000000001, reg_lambda=0.1, subsample=0.9;, score=0.850 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.2, colsample_bytree=0.2, gamma=1.5000000000000002, learning_rate=0.01, max_depth=11, min_child_weight=9, n_estimators=800, reg_alpha=1.9000000000000001, reg_lambda=0.1, subsample=0.9;, score=1.000 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.2, colsample_bytree=0.2, gamma=1.5000000000000002, learning_rate=0.01, max_depth=11, min_child_weight=9, n_estimators=800, reg_alpha=1.9000000000000001, reg_lambda=0.1, subsample=0.9;, score=0.950 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.30000000000000004, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.1, max_depth=5, min_child_weight=8, n_estimators=900, reg_alpha=0.5, reg_lambda=1.3000000000000003, subsample=0.2;, score=0.300 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.30000000000000004, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.1, max_depth=5, min_child_weight=8, n_estimators=900, reg_alpha=0.5, reg_lambda=1.3000000000000003, subsample=0.2;, score=0.350 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.30000000000000004, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.1, max_depth=5, min_child_weight=8, n_estimators=900, reg_alpha=0.5, reg_lambda=1.3000000000000003, subsample=0.2;, score=0.350 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.30000000000000004, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.1, max_depth=5, min_child_weight=8, n_estimators=900, reg_alpha=0.5, reg_lambda=1.3000000000000003, subsample=0.2;, score=0.350 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.30000000000000004, colsample_bytree=0.4, gamma=1.7000000000000002, learning_rate=0.1, max_depth=5, min_child_weight=8, n_estimators=900, reg_alpha=0.5, reg_lambda=1.3000000000000003, subsample=0.2;, score=0.300 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.6, colsample_bytree=0.8, gamma=0.5, learning_rate=0.01, max_depth=19, min_child_weight=6, n_estimators=650, reg_alpha=1.6, reg_lambda=1.4000000000000001, subsample=0.2;, score=0.300 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.6, colsample_bytree=0.8, gamma=0.5, learning_rate=0.01, max_depth=19, min_child_weight=6, n_estimators=650, reg_alpha=1.6, reg_lambda=1.4000000000000001, subsample=0.2;, score=0.600 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.6, colsample_bytree=0.8, gamma=0.5, learning_rate=0.01, max_depth=19, min_child_weight=6, n_estimators=650, reg_alpha=1.6, reg_lambda=1.4000000000000001, subsample=0.2;, score=0.350 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.6, colsample_bytree=0.8, gamma=0.5, learning_rate=0.01, max_depth=19, min_child_weight=6, n_estimators=650, reg_alpha=1.6, reg_lambda=1.4000000000000001, subsample=0.2;, score=0.500 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.6, colsample_bytree=0.8, gamma=0.5, learning_rate=0.01, max_depth=19, min_child_weight=6, n_estimators=650, reg_alpha=1.6, reg_lambda=1.4000000000000001, subsample=0.2;, score=0.350 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.2, colsample_bytree=1.0, gamma=1.4000000000000001, learning_rate=0.005, max_depth=1, min_child_weight=17, n_estimators=950, reg_alpha=1.8000000000000003, reg_lambda=0.7000000000000001, subsample=0.8;, score=0.350 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.2, colsample_bytree=1.0, gamma=1.4000000000000001, learning_rate=0.005, max_depth=1, min_child_weight=17, n_estimators=950, reg_alpha=1.8000000000000003, reg_lambda=0.7000000000000001, subsample=0.8;, score=0.350 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.2, colsample_bytree=1.0, gamma=1.4000000000000001, learning_rate=0.005, max_depth=1, min_child_weight=17, n_estimators=950, reg_alpha=1.8000000000000003, reg_lambda=0.7000000000000001, subsample=0.8;, score=0.350 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.2, colsample_bytree=1.0, gamma=1.4000000000000001, learning_rate=0.005, max_depth=1, min_child_weight=17, n_estimators=950, reg_alpha=1.8000000000000003, reg_lambda=0.7000000000000001, subsample=0.8;, score=0.350 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.2, colsample_bytree=1.0, gamma=1.4000000000000001, learning_rate=0.005, max_depth=1, min_child_weight=17, n_estimators=950, reg_alpha=1.8000000000000003, reg_lambda=0.7000000000000001, subsample=0.8;, score=0.300 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.4, colsample_bytree=1.0, gamma=1.7000000000000002, learning_rate=0.01, max_depth=19, min_child_weight=11, n_estimators=550, reg_alpha=1.7000000000000002, reg_lambda=1.8000000000000003, subsample=0.7000000000000001;, score=0.950 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.4, colsample_bytree=1.0, gamma=1.7000000000000002, learning_rate=0.01, max_depth=19, min_child_weight=11, n_estimators=550, reg_alpha=1.7000000000000002, reg_lambda=1.8000000000000003, subsample=0.7000000000000001;, score=0.800 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.4, colsample_bytree=1.0, gamma=1.7000000000000002, learning_rate=0.01, max_depth=19, min_child_weight=11, n_estimators=550, reg_alpha=1.7000000000000002, reg_lambda=1.8000000000000003, subsample=0.7000000000000001;, score=0.700 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.4, colsample_bytree=1.0, gamma=1.7000000000000002, learning_rate=0.01, max_depth=19, min_child_weight=11, n_estimators=550, reg_alpha=1.7000000000000002, reg_lambda=1.8000000000000003, subsample=0.7000000000000001;, score=0.850 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.4, colsample_bytree=1.0, gamma=1.7000000000000002, learning_rate=0.01, max_depth=19, min_child_weight=11, n_estimators=550, reg_alpha=1.7000000000000002, reg_lambda=1.8000000000000003, subsample=0.7000000000000001;, score=1.000 total time=   0.1s

[CV 2/5] END colsample_bylevel=0.1, colsample_bytree=0.5, gamma=0.5, learning_rate=0.05, max_depth=17, min_child_weight=18, n_estimators=950, reg_alpha=1.0, reg_lambda=0.9, subsample=0.6;, score=0.350 total time=   0.1s

[CV 1/5] END colsample_bylevel=0.1, colsample_bytree=0.5, gamma=0.5, learning_rate=0.05, max_depth=17, min_child_weight=18, n_estimators=950, reg_alpha=1.0, reg_lambda=0.9, subsample=0.6;, score=0.350 total time=   0.1s

[CV 3/5] END colsample_bylevel=0.1, colsample_bytree=0.5, gamma=0.5, learning_rate=0.05, max_depth=17, min_child_weight=18, n_estimators=950, reg_alpha=1.0, reg_lambda=0.9, subsample=0.6;, score=0.350 total time=   0.1s

[CV 4/5] END colsample_bylevel=0.1, colsample_bytree=0.5, gamma=0.5, learning_rate=0.05, max_depth=17, min_child_weight=18, n_estimators=950, reg_alpha=1.0, reg_lambda=0.9, subsample=0.6;, score=0.350 total time=   0.1s

[CV 5/5] END colsample_bylevel=0.1, colsample_bytree=0.5, gamma=0.5, learning_rate=0.05, max_depth=17, min_child_weight=18, n_estimators=950, reg_alpha=1.0, reg_lambda=0.9, subsample=0.6;, score=0.300 total time=   0.1s

Accuracy = 0.98