Giter VIP home page Giter VIP logo

writing.com-archival's Introduction

writing.com-archival

This is a Python3 utility for archiving interactive stories from writing.com.

Archiving a story will download every chapter from that story into ./archive/<story_id>/ as a set of browsable html files which you can open in your browser. Updating a story downloads any new chapters into the archive.

Command line usage:

$ cd <path-to-source> 
$ pip install -r requirements.txt          # Install dependencies
$ chmod +x run.py                          # If the following commands do not work
$ ./run.py get <id1> <id2>...              # Downloads or updates interactives with item_ids <id1>, <id2>...
$ ./run.py get_search "<url1>" "<url2>"... # Downloads every interactive in these search results. See note below.
$ ./run.py update                          # Update existing archives

Item_id?

The item_id for a story is in its url: http://www.writing.com/main/interact/item_id/$(THIS_IS_THE_ITEM_ID)/map/14411122

get_search

To get the proper URL, first perform a search on writing.com. Then click the magnifying glass icon just above the results to redo the search. Now your URL is in the correct form for get_search. Don't forget to quote it with "".

Your first use

You will be asked to enter your login credentials to writing.com on your first use. This is because writing.com requires a logged-in session to access interactive chapters. This won't save your password, but session cookies will be saved in a file called session.

Dependencies

Python3 and the packages in requirements.txt are required.

Troubleshooting

You may encounter errors while trying to download stories. If the error does not crash the utility, I suggest trying to download it again, as some of the errors are transient. But let me know what story/chapter you had trouble with and I'll see what I can do.

If the error does crash the utility, I suggest logging in again. Best case scenario either your login was unsuccesful or writing.com has decided to stop serving your session. Delete session (not session.py!) and you'll be asked for your credentials again. Note that it won't tell you if the login was successful or not. If it still fails, let me know.

writing.com-archival with Docker

This utility can also be run in a container. The container handles installing all the required dependencies for running the utility.

Requirements

Windows

Windows 7 / Windows 8 / Windows 8.1 / Windows 10 Home

Windows 10 Professional / Windows 10 Enterprise

MacOS

Linux

How to Use this Image

$ cd <path-to-source>
$ docker-compose build
$ docker-compose run writing.com-archival
# After running the above commands, the command line arguments stay the same.
root@writing-com-archival:/code# ./run.py get <id1> <id2>...              # Downloads or updates interactives with item_ids <id1>, <id2>...
root@writing-com-archival:/code# ./run.py get_search "<url1>" "<url2>"... # Downloads every interactive in these search results. See note below.
root@writing-com-archival:/code# ./run.py update                          # Update existing archives

Command Explanations

docker-compose build - This command will use the docker-compose.yml file to build the container defined inside the Dockerfile. docker-compose run writing.com-archival - This command will use the docker-compose.yml file to run the writing.com-archival container interactively in the current shell session. Do not use git for bash to run this command.

writing.com-archival's People

Contributors

ackater avatar bixoa avatar leeolder avatar solracht avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.