Giter VIP home page Giter VIP logo

tweetlr's Introduction

tweetlr

travis-ci    Coverage Status
Your chance to contrubute: Stories in Ready

tweetlr crawls twitter for a given term, extracts photos out of the collected tweets' short urls and posts the images to tumblr.

There is a new tweetlr "as-a-service" where you can easily create an account without having to know or host anything.

Supported image sharing services

tweetlr supports

  • instagram
  • twitter
    • photobucket
    • twimg
  • foursquare
  • path.com
  • twitpic
  • imgly
  • eyeem.com
  • t.co shortened links to pictures
  • every photo service accessible via embed.ly (see photo providers)

Installation

Use gem install tweetlr if you're using rubygems or add the line gem 'tweetlr' to your Gemfile if you're using bundler.

Configuration

It's essential that you have a directory called config in the directory you are starting tweetlr in, which has to contain the configuration file tweetlr.yml:

results_per_page: 100
result_type: recent
search_term: 'cat dog unicorn' #find tweets containing any of these terms
start_at_tweet_id: 61847783463854082 # the tweet id to start searching at
api_endpoint_tumblr: 'http://www.tumblr.com'
twitter_app_consumer_key: YOUR APPS TWITTER API KEY
twitter_app_consumer_secret: YOUR APPS TWITTER API SECRET
twitter_oauth_token: YOUR APPS TWITTER API OAUTH TOKEN
twitter_oauth_token_secret: YOUR APPS TWITTER API OAUTH TOKEN SECRET
tumblr_oauth_api_key: YOUR APPS TUMBLR API TOKEN
tumblr_oauth_api_secret: YOUR APPS TUMBLR API SECRET
tumblr_oauth_access_token_key: YOUR BLOGS OAUTH ACCESS TOKEN KEY
tumblr_oauth_access_token_secret: YOUR BLOGS OAUTH ACCESS TOKEN SECRE
tumblr_blog_hostname: YOUR BLOGS HOSTNAME #e.g. myblog.tumblr.com
embedly_key: '' #tweetlr uses http://embedly.com for link processing. a free plan containing an api key is available & recommended to use in order to ensure full support
update_period: 300 #check for updates every 300 secs = 5 minutes
shouts: 'says' # will be concatenated after the username, before the message: @mr_x says: awesome things on a photo!
loglevel: 1 # 0: debug, 1: info (default), 2: warn, 3: error, 5: fatal
whitelist: #twitter accounts in that list will have their tweets published immediately. post from others will be saved as drafts. blank list will publish all tweets immediately
  - whitey_mc_whitelist
  - sven_kr

Usage

Make sure you put the configuration file in it's proper place as mentioned above, then:

start/stop tweetlr using tweetlr start/tweetlr stop. Run tweetlr without arguments for a list of options concerning the daemon's options.

For a easy to modify working example, check out the tweetlr_demo.

Enjoy!

tweetlr's People

Contributors

5v3n avatar lennylinux avatar rubiii avatar waffle-with-pears avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

rubiii hughevans

tweetlr's Issues

twitpic integration doesn't work

twitpic seems to prevent the retrieval of the absolute image url via the api.

a possible solution: parse the html of the twitpic page showing the image.

Occasional `Encoding::CompatibilityError`

Every few weeks I see this error in the logs:

/home/5v3n/.rvm/gems/ruby-1.9.2-p180@tweetlr/gems/tweetlr-0.1.3/lib/tweetlr.rb:251:inrescue in http_get': incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)`

Which could be either a response in utf-8 that could not be read, or some "hidden utf-8 code block" in the ruby file that wasn't executed before.

rare curl http error to be rescued: Curl::Err::HostResolutionError

veeeery rare, but hurst nonetheless: twitter hostname resolving error:

.../ruby-1.9.2-p290@tweetlr/bundler/gems/tweetlr-05ed2f5739c6/lib/tweetlr.rb:253:in 'perform': Couldn't resolve host name (Curl::Err::HostResolutionError) from /home/5v3n/.rvm/gems/ruby-1.9.2-p290@tweetlr/bundler/gems/tweetlr-05ed2f5739c6/lib/tweetlr.rb:253:in 'http_get' from /home/5v3n/.rvm/gems/ruby-1.9.2-p290@tweetlr/bundler/gems/tweetlr-05ed2f5739c6/lib/tweetlr.rb:116:in 'lazy_search_twitter'

idea: collect all posted pics

Requested by @LasseL - who thinks it's a good idea to be able to collect all the pictures in a single download so you can do a giant poster.

I agreed, although I'd prefer a Moocards stickerbook :-).

soundcloud/tumblr connect

just found this post from the tumblr staff (http://staff.tumblr.com/post/6074690226/soundcloud). tumblr is now a sharing feature at soundcloud and posting soundcloud tracks as audio posts on tumblr is now possible.

with the soundcloud apps, you can record, edit metadata, publish to soundcloud and share to twitter in one go.

may be far fetched, but with pictures, video and audio, you had something like a decentralized reporting network in place...

Handle Tumblr photo post quota

When a tweetlr hits the tumblr photo quota wall, we currently save the tweet id in the TweetlrConfiguration.since_id field. If we would only save it for successful, non-quota-impeded tumblr photo posts we have a chance of catching/posting the pics the next day.

refactor code: decoupling & cohesion

right now, tweetlr grew into a direction where the "a tiny little script needs no decoupling"-state is not acceptable any more.

plan: decouple functionality & use a highly cohesive, module based approach

support for for video and links

videos (at least youtube, vimeo would also be nice) => post as video
links (i know, probably not as easy as it sounds, since pics and videos are basically links in the first place too...) => post as link

replace basic auth

basic auth is sufficiently secure for servers / hosted tweetlrs. but if you're using a laptop to stream your event in a free wifi, it's probably more cozy to use cookie auth.

error when trying out

tweetlr gives following error, with or without arguments:

/Library/Ruby/Gems/1.8/gems/tweetlr-0.0.6/bin/tweetlr:7: undefined method require_relative' for main:Object (NoMethodError) from /usr/bin/tweetlr:19:inload'
from /usr/bin/tweetlr:19

any ideas?

Add eyeem.com crawler

embed.ly doesn't support eyeem - we need a handcrafted crawler. see lib/processors/photo_services.rb and spec/processors/photo_services_spec.rb for examples of other services e.g. path and foursquare.

archive functionality

tweets are lost within hours. but they are just 140 bytes + metadata. since they are grabbed and analyzed by tweetlr anyways, why not dump them in a simple text file? or a ultra-light database table for later export?

Retweet: handle MT

Currently RT & quoted retweets are handled - we need MT in addition to it.

picplz nil bug

the picplz parser crashes due to nil objects - check for them to increase stability / continuity

Logger doesn't work w/ Daemons gem

Seems like Logger does not work when running tweetlr w/ the Daemons gem. This is due to Daemons redirecting all output to STDOUT / STDERR to a logfile itself.

Solution could be: do not use Logger but puts to STDOUT / STDERR. Which feels kind of sucky...

is it safe?

noticed that tumblr is not using https by default (neither for logging in or creating accounts) and was wondering how tweetlr is handling this. is my email-adress / password duo sent through the web in plaintext every five minutes?

idea: persist the lazy search ids

right now, you have to manually adjust the twitter timestamp / since_id each time you restart the crawler. persisting it (e.g. in a file) would take much ops hassle away.

tweetlr.tid: wrong tweet id persisted

we persist tweets.last['id'] which appears to be the oldest tweet collected. we have to use tweets.first['id']. a little counter-intuitive... but that's the tweet id we need.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.