Social Feed Manager (SFM) harvests social media data from multiple platforms' public APIs to help archivists, librarians, and researchers to build social media collections. More information about the project itself.

• Documentation: http://sfm.readthedocs.org
• Installing SFM: http://sfm.readthedocs.io/en/latest/install.html
• About the SFM project: http://gwu-libraries.github.com/sfm-ui
• Project updates: Follow @SocialFeedMgr on Twitter
• Discussion: sfm-dev
• Tickets: sfm-ui is used for all ticketing.

This is a re-architected version of an earlier Social Feed Manager which had been in use at GWU Libraries since 2012.

Social Feed Manager allows users to:

• define collections comprising sets of targeted accounts, keywords, and other search strategies appropriate to different platforms.
• authorize SFM to harvest data from platforms on the user's behalf.
• view collection information and metadata about harvests.
• extract, filter, and export the datasets to formats appropriate to the user's work.

The full Social Feed Manager is made up of several component repositories:

• sfm-ui: (this repo) User interface and datastore for collection and harvest information.
• sfm-flickr-harvester: A harvester for Flickr.
• sfm-twitter-harvester: A harvester for Twitter.
• sfm-weibo-harvester: A harvester for Sina Weibo.
• sfm-tumblr-harvester: A harvester for Tumblr.
• sfm-web-harvester: A harvester for web resources using Heritrix.
• sfm-utils: Utilities to support SFM.
• sfm-docker: Docker configuration for deploying SFM.
• sfm-elk: Proof-of-concept analytics dashboard using the ELK stack.

• Provides a user interface to set up Collection Sets, Collections, and Seeds
• Provides Django admin views to administer Credentials, Groups, and other model entities.
• Publishes harvest.start messages for flickr collections. The app schedules harvest.start messages for publication when the user updates an existing, active Collection.
• Includes a scheduler which uses apscheduler to schedule publication of harvest.start messages.
• Binds to harvest.status.*(.*) messages and creates a Harvest object (visible in the admin views) for each harvest status message received. The message consumer is started via the startconsumer management command.

Behind the scenes, SFM uses a set of carefully managed processes to harvest and and store this data, recording its actions in detail.

Tickets / pull requests / discussion are welcome.

cd sfm

./manage.py test --settings=sfm.settings.test_settings

Social Feed Manager is supported by a grant from the National Historical Publications & Records Commission.