Giter VIP home page Giter VIP logo

archiver's Introduction

👋 Hi | नमस्ते | Olá | Salut | Nǐ hǎo !

I'm Apurv Mishra, a programmer by passion and a Software Engineer at Capstone Investment Advisors. I'm also a hackathon addict and love working on random but interesting projects! When I'm not working/studying/programming you can find me roaming around London, exploring the different boroughs and meeting new people.

  • 🔭 I’m currently a Software Engineer at Capstone working on the Risk Technologies team.
  • Previously I worked at Stripe in the Bank transfers payment method team.
  • I studied Computer Science and graduated from University of Edinburgh in 2022.
  • 👨‍💻 I write lots of Python at work now.
  • 💬 Ask me about Python, hackathons, Indian cooking and (all)sports 😄.
  • ⚡ Where to find more of my work:
  • 📫 How to reach me:

Thank you for visiting my little place on Github! Hope you have an awesome day!

visitor count

archiver's People

Contributors

apurvmishra99 avatar deepsourcebot avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar

Forkers

gabldotink

archiver's Issues

15 pages per minute

the script should slow down considering that you can only upload 15 pages per minute, otherwise you get blocked for 5 minutes

Capture doesn't actually capture a website. If it isn't already archived

if you look at line

archiver/archiver/archive.py

Line 71 in 07207f6

(link, self.sync_get(self.SAVE_ENDPOINT + link))

you'll see that this just probes archive.org to know if the site exists, but it doens't actually tell archive.org to archive it. You need to send a POST request like this

curl "https://web.archive.org/save/https://example.com/ ^
  -H "authority: web.archive.org" ^
  -H "origin: https://web.archive.org" ^
  -H "content-type: application/x-www-form-urlencoded" ^
  -H "accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9" ^
  -H "sec-fetch-site: same-origin" ^
  -H "sec-fetch-mode: navigate" ^
  -H "sec-fetch-user: ?1" ^
  -H "sec-fetch-dest: document" ^
  -H "referer: https://web.archive.org/save/" ^
  --data-raw "url=https^%^3A^%^2F^%^2Fexample.com^%^2F&capture_outlinks=on^&capture_all=on"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.