Giter VIP home page Giter VIP logo

twitter-crawler's Introduction

Twitter-Crawler

Streaming APIs give us low latency access to Twitter’s global stream of Tweet data in almost real-time manner. This code is implemented with respect to tweepy library.

-Then, Paste the (consumer_key, consumer_secret, access_token and access_secret) tokens in config.py with your generated configuration tokens from twitter developer website.

-Make sure to create a file to save the crawled data in, by using command (mkdir data).

-To run the code : use the following command by passing your requested keywords to crawl lively tweets about it. python twitter_stream_download.py -q (your keyword) -d data

example: If you want to crawl up-to-dates tweets about e-learning, then (python twitter_stream_download.py -q #e-learning -d data) has to be run using your cmd.

Dataset

You can download the dataset (Tweets for keyword 'MOOC/s' over the period 3 to 31 of July 2017 resulting in more than 43,000 full structured time-stamped tweets). MOOC: stands for Massive Open Online Courses.

https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi%3A10.7910%2FDVN%2FFJXK56)

  • Tweets for #MOOC/s over the period 3 to 31 of July 2017 resulting in more than 43,000 full structured time-stamped tweets.
  • #GameOfThrones hashtag on the third day of releasing (Jul-18 2017) the 7th season, results a 4447 tweet within only 16 minutes!

(This meet one of our hypothesis which assumes that social media users are more interested of being updated about entertainment contents over educational matterial).

Figures

Linkedin post : a figure shows a post at Linkedin platform, which the post contains an educational video material. img

Linkedin post activity: result of the engagement at the Linkedin post over the last month (September). img

Twitter Post: a figure shows a tweet posted at twitter platform over a month. The tweet contains a link for same educational video that Linkedin have. img

Twitter post activity: result of posting this tweet for over a month, contains counter for how many likes, link clicks and hashtag clicks. img

2g_3g_4g_coverage_map: containts a map that address all countries in the world with the current internet network coverage type (2G: Red, 3G:Blue and 4G:Yellow). *http://extensia-ltd.com/africa-africa-world-looks-like-2g-3g-4g/ img

MOOCs global covereage by HPI: Learning analysis shows less usage of elearning platforms in Africa compared to the penetration and the popularity of social networking sites worldwidely. img

Three_Hashtags _Visualization: a hashtag network for (#OpenHpi, #HPI_DE, #OpenSAP) over the period 20th-27th/June/2017.
img

Internet Service: This table and diagram shows the increasing Internet Services in the East African country with high penetration of mobiles such as mobile devices and wireless communication. img

Overall usage of Social Media Applications Figure summarizes the overall usage of social media applications* with frequency use of Social Media among respondent. *Source: Deo Shao , OSN tanzania. img

Facebook Free Mode Figure shows users of Facebook have an option to go "FREE MODE" which offer free Internet data services to the user in Facebook. img

Service Provider Facebook Bundles Figure shows a typical menu from the ISP showing an optional selection of bundle bundles explicitly for Social media such as WhatsApp, Facebook and YouTube. img img

twitter-crawler's People

Contributors

raadbintareaf avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.