CODEWITHSUNDEEP
machine-learningdecision-treelightgbm
Jame
Jame

Reputation: 99

what is the meaning of bins of lightgbm?

I am using lightgbm to do the prediction competition recently, it is my first time using this decision tree model, and I am a beginner of machine learning, when I train the model to fit the data, I got the results below:

[LightGBM] [Info] Total Bins 3499
[LightGBM] [Info] Number of data: 595192, number of used features: 25
/Users/Jame/anaconda3/lib/python3.6/site- 
packages/lightgbm/basic.py:725: UserWarning: categorical_feature in 
param dict is overridden.
warnings.warn('categorical_feature in param dict is overridden.')
[LightGBM] [Warning] Unknown parameter: vebose
[1] valid_0's multi_logloss: 2.35527
 Training until validation scores don't improve for 50 rounds.
[2] valid_0's multi_logloss: 2.31477
[3] valid_0's multi_logloss: 2.27614
[4] valid_0's multi_logloss: 2.23926
[5] valid_0's multi_logloss: 2.20397
[6] valid_0's multi_logloss: 2.16997
[7] valid_0's multi_logloss: 2.1372
[8] valid_0's multi_logloss: 2.10566
[9] valid_0's multi_logloss: 2.07528

as you can see from the first line, what is the Bins means in lightgbm, where can I get the detail, go to the paper fo lightgbm??

Thanks a lot!

Upvotes: 4

Views: 5586

Answers (3)

xingpei Pang
xingpei Pang

Reputation: 1285

Binning is a technique for representing data in a discrete view(histogram). Lightgbm uses a histogram based algorithm to find the optimal split point while creating a weak learner. Therefore, each continuous numeric feature (e.g. number of views for a video) should be split into discrete bins. enter image description here

enter image description here

Also, in this GitHub repo, you can find some comprehensive experiments which completely explains the effect of changing max_bin on CPU and GPU. enter image description here

enter image description here

If you define max_bin 255 that means we can have a maximum of 255 unique values per feature. Then Small max_bin causes faster speed and large value improves accuracy.

Upvotes: 4

fangorn
fangorn

Reputation: 34

As a beginner, i don't think you need to think a lot about the max_bin parameter.

LightGBM optimizes the dataset storage depending on the binary power of the parameter max_bin. For instance, max_bin = 255 allows to uses 8 bits to store a single value. max_bin = 63 would require only 6 bits, while max_bin = 15 would require only 4 bits. Such optimization allows to train large datasets with a small memory footprint, such as Higgs 10M with 63 bins on a GPU using only 611MB RAM. Defaults to 255 (defaults to 16 for GPU).

Upvotes: 1

Ugur MULUK
Ugur MULUK

Reputation: 464

It actually refers to the parameter max_bin. The parameter controls how your data is partitioned into bins before going into learning. It is 255 as default, I’d recommend you to tune this parameter lastly. This parameter has more to be explained yet it would be irrelevant with your question.

The verbose that you ask about simply tells you how many data bins there are, before going into training the model.

Upvotes: -1

Related Questions