• folder jupyter contains jupyter notebooks used to analyze the data
• folder sources contains scripts used to load the data from various sources
• folder sripts contains helper scripts to manage the project

• Postgresql 9.5
• Python 3.5

in more details https://developer.github.com/v3/activity/events/types/#pushevent

Data sources for more information

• https://github.com/npm/download-counts -> getting download counts for particular package
• http://registry.npmjs.org/-/all -> returns all packages on NPM roughly 250 000 and size of the json is 175 mbs
• http://registry.npmjs.org/{package-name} -> returns json for specific package
• http://npm-stat.com -> has download stats about packages, probably just UI built on top of https://github.com/npm/download-counts
• http://www.npm-stats.com -> same as above
• github API -> contains metadata about packages
• github archive
• http://ghtorrent.org -> mirror data from Github REST API

List of what I have been able to get so far

• stargazers count -> github API
• forks count -> github API
• open issues count -> github API
• size -> github API
• subsribers count -> github API
• number of dependencies -> npm API
• mainteners counter -> npm API
• contributors counter -> npm API
• has readme -> npm API
• has stable release -> npm API
• has tests -> npm API
• downloads count -> npm download API
• has linter -> ?
• code complexity -> ?
• issues, how fast they are attended and closed -> github archive
• commits, frequency and size -> github archive

• look at popular packages what examine what features makes them popular
• try to split packages into 2 groups at first and classify what group it belongs to
• try basic classification algorithms to see whether we will get an improvement over random guessing, 5% to 10% is already a good result
• a way to split packages would be to look at mean of popularity feature and variance, and have packages that are above and below this
• they also mentioned that noisy data is expected