minerva-ml / open-solution-home-credit Goto Github PK

https://www.kaggle.com/jsaguiar/lightgbm-with-simple-features-0-785-lb/code
+ fork : https://www.kaggle.com/poohtls/fork-of-fork-lightgbm-with-simple-features/code

When validation, loaders, make_submit function, and model training steps (random forest and sv regression) are ready:

• build first pipeline from these pieces
• run grid search
• submit to Kaggle :)

so that it has paths to data

At present, Target Encoding for categorical variables is used (http://contrib.scikit-learn.org/categorical-encoding/targetencoder.html). It shall be discussed whether it is an optimal encoding in the presence of e.g. LigthGBM built-in mechanisms.

http://jinja.pocoo.org/

Jinja2 is a full featured template engine for Python

• investigate age buckets
• check for important dates in Spain (Studies, retirement)

• Use new adapter syntax
• run experiment in order to make sure that it works correctly.
• assume that steppy-toolkit is pip installable

• look for useful features
• tricks
• models
• validation strategies, etc.

• test with / without neptune
• write Wiki documentation
• write info on Kaggle
• report on results

• create features based on EXT_SOURCE_1,2,3
• sum
• average (weighted perhaps)

• XGBoost
• LogReg

Check their implementation in TalkingData and port to steppy-toolkit

https://www.kaggle.com/codename007/home-credit-complete-eda-feature-importance

• outliers
• mislabeling

Look for features. Good ideas deserve Issue on our GitHub.

https://www.kaggle.com/shanth84/credit-card-balance-feature-engineering/

• Analyze dataset to identify good candidates for simple features
  • simple, direct, easy to implement with minimal effort.
• Implement steppy Transformer that prepares features for sklearn regression algorithms:
  • random forest
  • support vector regression
  • others(?)

check this code:
https://community.neptune.ml/t/htop-for-cpu-and-memory-inspection/334

• this is preparations Step
• use our steppy lib. -> ask @pknut or @mromaniukcdl for help, as needed.

https://www.featuretools.com/

research if its useful:
http://scikit-learn.org/stable/auto_examples/ensemble/plot_partial_dependence.html

1st level:

• rf
• svc
• xgb
• lightGBM

How should we handle NaNs?

• remaining data is data without train/test IDs
• write down your ideas as a comment to this Issue

• how it works?
• which parameters are important and why?
• rules of thumb for grid search

use steppy-toolkit

Write here: LightGBM-and-basic-features

Use DSB wiki as a reference.

When #4 is done:

• implement steppy transformer that train random forest
• implement steppy transformer that train support vector regression
• handle NaN

n fold CV

• inside single step?
• or n steps for n-fold CV?