Comments (1)
Thanks for using LightGBM.
I've reformatted your post a bit to make the difference between code, output from code, and your own words a bit clearer. If you're new to GitHub and/or writing markdown, please see https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#quoting-code to learn about how I did that.
I don't immediately see what's wrong with your model file but in general... don't manually merge files like this. There are just too many ways for it to go wrong (mixed line endings, accidental inclusion of special characters, failure to update all parameters correctly, etc.).
If it's acceptable to not be training both models at the same time, do this:
- train the first model
- save it to a text file with
.save_model()
- move that file over to the other machine
- train another model on that machine, with new data, passing the path to the first model file to
init_model
That will cause LightGBM to begin boosting from the end of the first model, using your new dataset. It's not exactly the same as the process you described (where you train two models from scratch and then combine them), but it's a good way to incorporate learnings from new data into an already-training model.
Like this pattern:
LightGBM/tests/python_package_test/test_engine.py
Lines 1050 to 1063 in 28536a0
There are some more details on how to do this at https://stackoverflow.com/questions/73664093/lightgbm-train-vs-update-vs-refit/73669068#73669068
There is an API in LightGBM's C API, LGBM_BoosterMerge()
, that I believe can be used to take 2 models and combine them together without training, but it's not currently exposed in the Python package.
Lines 2008 to 2009 in 28536a0
We could consider adding that in the Python package.
from lightgbm.
Related Issues (20)
- [Question] Setting values for linear coefficient HOT 5
- LGBM_BoosterUpdateOneIterCustom requires objective_func_ not null HOT 2
- [R-package] Expose start_iteration to dump/save/lgb.model.dt.tree HOT 1
- [GPU] Kernel crashed when using GPU HOT 5
- [python-package] LGBM hangs with high number of categories HOT 2
- [python-package] Early stopping callback added when early_stopping_round = 0 HOT 15
- [python-package] `init_score` causes the predictions different HOT 6
- Classifier predict numerical precision issue with large raw_score HOT 4
- score belong to different target classes. class 0 when init_score is given else 1 HOT 2
- LGBMClassifier gives non-deterministic outputs with very low AUC score compared to xgboost and catboost HOT 2
- [R-package] lgb.cv() fails with categorical features HOT 3
- [python-package] Can't continued training on Dataset with SequenceDataset(lgb.Sequence)
- Q:error in predict
- [python-package] CUDA version not truly installing HOT 4
- Continue traing by C_API HOT 2
- Failed to build by docker HOT 3
- Memory not being returned to OS on calling C.LGBM_BoosterFree HOT 15
- checkpoint can't be loaded on MacOS with M3 HOT 12
- [python-package] First metric is not used for early stopping
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lightgbm.