Giter VIP home page Giter VIP logo

waybacktweets's Introduction

πŸ›οΈ Wayback Tweets

Streamlit App GitHub release (latest by date including pre-releases)

Tool that displays, via Wayback CDX Server API, multiple archived tweets on Wayback Machine to avoid opening each link manually. The application is a prototype written in Python with the Streamlit framework and hosted on Streamlit Cloud, allowing users to apply filters based on specific years and view tweets that lack the original URL.

Community

"We're always delighted when we see our community members create tools for open source research." β€” Bellingcat

"#myOSINTtip Clarissa Mendes launched a new tool for accessing old tweets via archive.org called the Wayback Tweets app. For those who love to look deeper at #osint tools, it is available on GitHub and uses the Wayback CDX Server API server (which is a hidden gem for accessing archive.org data!)" β€” My OSINT Training

"Original way to find deleted tweets." β€” Henk Van Ess

"This is an excellent tool to use now that most Twitter API-based tools have gone down with changes to the pricing structure over at X." β€” The OSINT Newsletter - Issue #22

"One of the keys to using the Wayback Machine effectively is knowing what it can and can’t archive. It can, and has, archived many, many Twitter accounts... Utilize fun tools such as Wayback Tweets to do so more effectively." β€” Ari Ben Am

"Want to see archived tweets on Wayback Machine in bulk? You can use Wayback Tweets." β€” Daily OSINT

"Untuk mempermudah penelusuran arsip, gunakan Wayback Tweets." β€” GIJN Indonesia

"A tool to quickly view tweets saved on archive.org." β€” Irina_Tech_Tips Newsletter #3

Development

Requirement

  • Python 3.8+

Installation

$ git clone [email protected]:claromes/waybacktweets.git

$ cd waybacktweets

$ pip install -r requirements.txt

$ streamlit run app.py

Streamlit will be served at http://localhost:8501

Changelog

Check out the releases.

Todo (2024 planning)

  • Code review
  • UX review (filter before requesting)
  • Add a calendar interface (Wayback Machine timestamp)
  • Prevent duplicate URLs/Review the "Unique tweets" option
    • Counters
    • Collapsing
  • Sorting in ascending and descending order
  • Download dataset
  • Fix parse_links exception
  • Update Streamlit version
  • Add metadata information
  • Parse MIME types: warc/revisit, text/plain, application/http
  • Documentation: Explain the mapping of archived URLs and the parsing process
  • Create CLI
  • Pagination
    • Footer
    • Disabled/Empty states
  • Feedback
  • Review data cache
  • Changelog
  • Define range size by user
  • Filter by period/datetime
  • Add contributing guidelines

Contributing

We welcome contributions from everyone, whether it's through bug reporting, feature suggestions or code contributions.

If you need help, or have ideas on improving this app, please open a new issue or reach out to [email protected].

Acknowledgements

  • Tristan Lee (Bellingcat's Data Scientist) for the idea of the application.
  • Jessica Smith (Snowflake's Marketing Specialist) and Streamlit/Snowflake teams for the additional server resources on Streamlit Cloud.
  • OSINT Community for recommending the application.

Note

If the application is down, please check the Streamlit Cloud Status.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.