Giter VIP home page Giter VIP logo

drivendata_submission's Introduction

Submission to "Pump it Up: Data Mining the Water Table" competition on DrivenData

Preprocessing techniques

  1. Imputed missing values in the following categorical columns with the most frequent value in each of them: 'funder', 'installer', 'subvillage', 'public_meeting', 'scheme_management', 'scheme_name', 'permit'.
  2. Imputed 0 values in construction_year with the earliest value of construction year.
  3. Imputed longitude and latitude values of records whose longitude were below 29, with the mean values of longitude and latitude when grouped by region_code. The reason was that according to the GPS coordinates of Tanzania (-6.3728253, 34.8924826), longitude values must be over 29. (Refer here)
  4. Imputed date_recorded values whose years were before construction_year, with the latest value of date_recorded.

All Imputation values were based on training data. Test data were transformed using those imputation values.

Feature Engineering techniques

  1. Created a new feature called age representing how old a waterpoint is, by subtracting the year of date_recorded from construction_year.
  2. Separated out year and month as year_recorded and month_recorded respectively, from date_recorded.
  3. Excluded columns id, date_recorded, recorded_by, num_private from training features.
  4. Applied Label (Integer) Encoding on categorical features. Encodings were based on training data. Unknown values in test data were labelled as 0 using a custom function.

Proof of submission

Sub_Proof

Final Rank

641

Rank_Proof

drivendata_submission's People

Contributors

bitrondo avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.