Giter VIP home page Giter VIP logo

twitterbigdataproject's Introduction

# tweet entry structure -------------------------------------------------------#
# -----------------------------------------------------------------------------#
contributors : None
truncated : False
text : @melvin622 j'vais te frapper
is_quote_status : False
in_reply_to_status_id : None
id : 708025430079627264
favorite_count : 0
source : <a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>
retweeted : False
coordinates : None
timestamp_ms : 1457641391664
entities : {u'user_mentions': [{u'id': 553012932, u'indices': [0, 10], u'id_str': u'553012932', u'screen_name': u'melvin622', u'name': u'Melvin 2.0'}], u'symbols': [], u'hashtags': [], u'urls': []}
in_reply_to_screen_name : melvin622
id_str : 708025430079627264
retweet_count : 0
in_reply_to_user_id : 553012932
favorited : False
user : {u'follow_request_sent': None, u'profile_use_background_image': True, u'default_profile_image': False, u'id': 1028993431, u'verified': False, u'profile_image_url_https': u'https://pbs.twimg.com/profile_images/681762263359995904/D2l3Azk3_normal.jpg', u'profile_sidebar_fill_color': u'DDEEF6', u'profile_text_color': u'333333', u'followers_count': 528, u'profile_sidebar_border_color': u'C0DEED', u'id_str': u'1028993431', u'profile_background_color': u'C0DEED', u'listed_count': 0, u'profile_background_image_url_https': u'https://abs.twimg.com/images/themes/theme1/bg.png', u'utc_offset': None, u'statuses_count': 5756, u'description': u'Samy Seghir / Coch /Lea / POLOGNE', u'friends_count': 693, u'location': u'lens ', u'profile_link_color': u'0084B4', u'profile_image_url': u'http://pbs.twimg.com/profile_images/681762263359995904/D2l3Azk3_normal.jpg', u'following': None, u'geo_enabled': False, u'profile_banner_url': u'https://pbs.twimg.com/profile_banners/1028993431/1452709651', u'profile_background_image_url': u'http://abs.twimg.com/images/themes/theme1/bg.png', u'name': u'M', u'lang': u'fr', u'profile_background_tile': False, u'favourites_count': 2816, u'screen_name': u'kazmierczak13', u'notifications': None, u'url': None, u'created_at': u'Sat Dec 22 18:03:32 +0000 2012', u'contributors_enabled': False, u'time_zone': None, u'protected': False, u'default_profile': True, u'is_translator': False}
geo : None
in_reply_to_user_id_str : 553012932
lang : fr
created_at : Thu Mar 10 20:23:11 +0000 2016
filter_level : low
in_reply_to_status_id_str : None
place : None
# -----------------------------------------------------------------------------#

twitterbigdataproject's People

Contributors

laucianexones avatar boorockt avatar

Watchers

 avatar Loni avatar Goraneza avatar  avatar

twitterbigdataproject's Issues

web sitesi

  • web sitesini github üzerinden yapmaca..

getCredentials.py

credentials dosyasını okuma tek bir merkezden yapılmalı.
getCredentials diye bir class oluştur, her scriptin başında bu kodu çağır
getCredentials ile api bağlantısını kontrol et, eğer düzgün bağlanmadı ile exception ile hata yakala, olmadı, yapamadım, yapamıyorum de..

encode-decode problemleri

türkçe karakterlerde saçma sapan encode decode edemedim diyor..

[kerem@leylek TwitterBigDataProject]$ ./searchHashTag.py Traceback (most recent call last): File "./searchHashTag.py", line 7, in <module> tweets = getCredentials.api.GetSearch('#AbdülHamidiAnlamak', count=100) File "/usr/lib/python2.7/site-packages/twitter/api.py", line 364, in GetSearch json = self._RequestUrl(url, 'GET', data=parameters) File "/usr/lib/python2.7/site-packages/twitter/api.py", line 3597, in _RequestUrl url = self._BuildUrl(url, extra_params=data) File "/usr/lib/python2.7/site-packages/twitter/api.py", line 3446, in _BuildUrl extra_query = self._EncodeParameters(extra_params) File "/usr/lib/python2.7/site-packages/twitter/api.py", line 3500, in _EncodeParameters return urllib.urlencode(dict([(k, self._Encode(v)) for k, v in parameters.items() if v is not None])) File "/usr/lib/python2.7/site-packages/twitter/api.py", line 3482, in _Encode return unicode(s).encode('utf-8') UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 4: ordinal not in range(128)

streamTimeline ile büyük bir dosya oluştur

  • tam olarak hangi parametreleri okumak istiyorum, API dokuya bak. bir sürü gereksiz parametre geliyor, boş dönen tweet entrylerini filtrele.
  • windows IDLE IDE ile okuduğum timeline da unicode karakterler düzgün decode edilmişti. Linux ile okuduğum dosyalar decode edilemedi..niye? (/data/ altında örnek dosyalar var)
  • amazon ec2 de 7.8 Gb boş yer mevcut. 1-2 gblik bir deneme dosyası yeterli olacaktır.

MongoDB

toplanan verileri mongoDB ye import etmece?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.