How to use XGBoost algorithm for regression in R?

Question

I was trying the XGBoost technique for the prediction. As my dependent variable is continuous, I was doing the regression using XGBoost, but most of the references available in various portal are for classification. Though i know by using

objective = "reg:linear"

we can do the regression but still I need some clarity for other parameters as well. It would be a great help if somebody can provide me an R snippet of it.

@ Amarjeet: do you actually receive an error when you try to run it or is your question rather how to tune the parameters? — Ruthger Righart, Oct 21 '15 at 07:44
@Amarjeet any updates on the subject ? I have the same problem. — deltascience, Oct 29 '16 at 14:49

score 5 · Answer 1 · edited Aug 12 '16 at 00:57

5

xgboost(data = X, 
        booster = "gbtree", 
        objective = "binary:logistic", 
        max.depth = 5, 
        eta = 0.5, 
        nthread = 2, 
        nround = 2, 
        min_child_weight = 1, 
        subsample = 0.5, 
        colsample_bytree = 1, 
        num_parallel_tree = 1)

These are all the parameters you can play around with while using tree boosters. For linear booster you can use the following parameters to play with...

xgboost(data = X, 
        booster = "gblinear", 
        objective = "binary:logistic", 
        max.depth = 5, 
        nround = 2, 
        lambda = 0, 
        lambda_bias = 0, 
        alpha = 0)

You can refer to the description of xg.train() in the xgboost CRAN document for detailed meaning of these parameters.

edited Aug 12 '16 at 00:57

arun

9,435
5
48
67

answered Oct 19 '15 at 09:24

Gaurav

1,441
1
11
28

1

I know its a very broad question to ask for, but if any specific answer related to the regression would be useful to understand. – Amarjeet Oct 19 '15 at 10:30
Linear regression and binary logistic regression are most common used methods with xgboost package... all the parameters in xgboost are to manipulate only the boosting part of the algo... there isn't much scope in the package to manipulate the regression technique... its either ols regression or binary logistic... – Gaurav Oct 19 '15 at 10:35

score 4 · Answer 2 · answered Oct 19 '15 at 19:21

The best description of the parameters that I have found is at

https://github.com/dmlc/xgboost/blob/master/doc/parameter.md

There are many examples of using XGBoost in R available in the Kaggle scripts repository. For example:

https://www.kaggle.com/michaelpawlus/springleaf-marketing-response/xgboost-example-0-76178/code

How to use XGBoost algorithm for regression in R?

2 Answers2