Giter VIP home page Giter VIP logo

rss_webscraper's Introduction

RSS_WebScraper

A small tool for fixing my social media addiction.

Follow only people you want without being tracked by third parties or distracted by addicting algorithms

Supports:
Picuki (Instagram)
Nitter (Twitter/X)
ProxiTok (Tiktok)
Invidious (YouTube)

Installation:

  1. Get the release from here
  2. Extract it into any folder
  3. Put config.json next to the binary (template here):
  • url - url of the local rss server
  • ffmpeg_location - location of ffmpeg (download here)
  • curl_impersonate_script_location - special version of curl, path to one of the browser scripts (eg. ff109) (download here)
  • scrape_timer - how many minutes it take to rescrape
  • nitter_instance - listed here, default nitter.net
  • proxitok_instance - listed here, default proxitok.pabloferreiro.es
  • invidious_instance - listed here, default invidious.poast.org
  • invidious_filter_shorts - true if you don't want shorts from followed channels in your feed, otherwise false
    • scrapes all accounts you subscribed to on the specific instance
    • to make it work, get your SID cookie value (Press F12 on the instance page -> Storage -> Cookies) and copy it into cookies/invidious.txt in format SID=value_of_the_cookie.
  • sites_and_usernames - put all accounts you want to scrape to the list. Picuki_stories_blacklist ignores stories and nitter_replies_blacklist ignores user replies
  1. Run the start.sh script (ideally on system boot)
  2. After scraping, all rss urls are located in rss_urls.txt

rss_webscraper's People

Contributors

dlabaja avatar

Stargazers

Charlie avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.