Gradient boosting is an ensemble technique using which multiple weak learners can be ensembled to create a stronger prediction model. These weak learners are decision trees. And these decision trees are used sequentially so that one decision tree can be built based on the error made by the previous decision tree.

We can use gradient boosting for both regression and classification problems. And this technique can be used to improve the accuracy of prediction. Interested readers who want to know more about gradient boosting, may want to refer to the following links:

https://www.youtube.com/watch?v=3CC4N4z3GJc
https://www.youtube.com/watch?v=2xudPOBz-vs
https://www.youtube.com/watch?v=jxuNLH5dXCs
https://www.youtube.com/watch?v=StWY5QWMXCw

Gradient Boosting Classifier using sklearn in Python

We can use the following Python code to solve a classification problem using gradient boosting.

from sklearn.model_selection import KFold
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import GradientBoostingClassifier
import pandas

data = pandas.read_csv("diabetes.csv")
D = data.values
X = D[:, :-1]
y = D[:, -1]

kfold = KFold(n_splits=10, shuffle=True, random_state=1)
model = GradientBoostingClassifier(n_estimators=100, random_state=1)

result = cross_val_score(model, X, y, scoring="accuracy", cv=kfold)
print(result.mean())

Here, we are first reading the Pima Indians Diabetes dataset. The dataset contains various predictor variables such as the …