4pr0n / ripme Goto Github PK

Examples:

http://yakovlev-vad.deviantart.com/art/strange-plant-412293638
Expands properly without having to touch the "Download" button, so RipMe downloads it without a hitch and in the original resolution.

http://csimadmax.deviantart.com/art/glitterface-344240007
This however, does not. Even though they are both .PNG's, and this picture is smaller in both vertical and horisontal resolution, Deviantart refuses a proper expansion into the original picture without clicking the "Download" button to the side. RipMe downloads the smaller "preview" instead of the original.

Edit: So it turns out this is a limitation that can be set in place by uploaders on a picture-by-picture basis. I suspect the fix required to make the downloads consistent would have to be that RipMe takes the picture from the link the "Download" button provides?

Edit2: My god, turns out they also have a limit on how much you can download before the downloadbutton stops working, this might be a lost cause.

Instant feedback would be nice.

Whenever I start up the program I'm hit immediately with this error:

After the closing the window the program proceeds to function normally. I can enter the url and change the settings. However once I click rip the same error comes up:

I tried running it normally and with admin rights through the command prompt, both with the same result. What do you think could be the issue? Let me know if I can help in anyway. Thanks!

There's a bug with the tumblr ripper (at least) that trims the url after the first space (if any). And then it tries to download the trimmed tag instead of the original one.

One example would be http://naughtynicegirl69.tumblr.com/tagged/my%20gif
Expected behaviour:

• Create folder in rips location named "tumblr_naughtynicegirl69_tag_my_gif"
• Download all images within that tag "my gif".

Current behaviour:

• Create folder in rips location named "tumblr_naughtynicegirl69_tag_my" (trimmed to the space)
• Download all images within the trimmed tag "my"

This happens whether the space in the given url is "%20" or an actual space " "

Should be fixed with something like:

http://stackoverflow.com/questions/2793940/why-right-click-is-not-working-on-java-application

Some albums° can't be ripped fully. Stop after ~900.

Right now they are numbered starting at 001 for the newest piece in a gallery, and while it is a really minor problem, I could see it being pretty nice having them numbered starting with the last (oldest) picture.

Thoughts?

While retries are probably good for some errors, it seems fairly plain that a 404 error should be considered non-recoverable. However, ripme continues to retry, up to the retry limit, for non-existant files. This feels like a mistake.

Ripping images/gifs from i.rarchives.com user pages is not working. The rip doesn't start. Example: http://i.rarchives.com/?url=rollmeup

Remove empty directory if nothing was downloaded / album is empty, etc.

Something like a ding or something, so you know when it's done, something like how the Firefox extension "DownThemAll" does it

Original request to allow duplicate URLs: #29

@Opzon

Easy dupe finder says I have 1k dupe files in the rip folder. I download from gonewild girls. http://gyazo.com/bcd59eba5c75c019465ae3e254bc3a0a of the dupelicate finder. (If I remove them and rerip they will just get downloaded again)

First of all, this tool is awesome. If there's anywhere I can donate to the developer, please let me know. I'd be delighted to do so. Thanks for all the effort you put in.

Feature request would be to have some method to schedule rerip all. I poked at it for a couple of minutes, but my lack of code knowledge made it too challenging for me. I was thinking I might be able to call the feature from a script on my machine, but couldn't figure that out either.

Thanks!

Example: http://www.imagefap.com/pictures/4649440/Frozen-(Elsa-and-Anna)
Make folder name "Frozen (Elsa and Anna)" instead of "imagefap_4649440".
Same with imgur and where applicable.

Alternatively: "Frozen (Elsa and Anna) [imagefap_4649440]", or a setting to change the behaviour.

The same thing might be applied to filenames (imgur gallery images sometimes have captions).
Imgur example: http://imgur.com/a/bXQpH

To preserve the original web-link names, maybe return the per-rip logs from the web-version of the ripper for every rip with lines like
Retrieved: Original URL
Saved as: rips\album caption\01_file caption

Also a reminder to sanitize caption inputs from captions that will break the filesystem ( \ / ? * " < > | : for NTFS) if this issue is to be considered.

Hey There,

Props for this awesome little program. I'm having a problem where when using the Re-rip all command (for about 86 Reddit user links) the program will hang at some point. It seems to be related to the Log view as when it crashes the bottom of the Log view goes squint (see image). If I close the Log view by clicking on the button whenever it pops ups I can successfully rip the entire list.

Cheers!

Most of the album will be correctly downloaded, but a couple of images are named like 013_image.php (which seems an odd choice for a default extension for downloads anyway). Changing extension to .jpg shows that the picture was properly downloaded, just not renamed.

Possibly a thread locking or concurrency issue?

I'm on Windows 7 btw

Examples: Facebook, Twitter, Instagram, Tumblr

These sites have some content that's only accessible from specific accounts.

ripme fetch images ending with h instead of the original one
Problems is that h is smaller resolution[1]

log
http://0bin.net/paste/DrMqm9pEYQTzRJ3+#wRLWhmMQ9fufGXUV72rSyqBzD9+lEXtaAHWX/qb/P7M=
sfv diff
http://cryptb.in/Z2#c7612971639c0e80daa22fb669fb2b72

[1] http://api.imgur.com/models/image

There are 6 total thumbnails that an image can be resized to. Each one is accessable by appending a single character suffix to the end of the image id, and before the file extension. The thumbnails are:
...
h Huge Thumbnail 1024x1024 Yes"

When ripping large imgur albums (eg imgur.com/r/wallpapers ), (several hundred images), the full album is often not fully ripped. Presumably related to imgur page loading (56 images/page) and/or the occasional "load more images".
This may be related to ripping on relatively low bandwidth connection.
If re-rip is attempted with overwrite disabled, sometimes the successive rip attempts will rip additional imgur pages.

PLEASE include an example URL of the type of album to rip.

Supported sites + commits:

• imgur (album, userpages, subreddits) - 04ce12b e904a7e
• instagram - e2bb412
• deviantart - b662b46
• "gonewild" - c48ebff
• twitter - 4a47cc6
• motherless - 1cdec76
• reddit - c8cfc57
• tumblr - c288afd
• 4chan and other *chans - Supported in cfb99e0
• vk.com (images, albums, video, and video albums) d51ad48 e5906db
• kinkyshare.com - Supported in 8db6e2a
• i.rarchives.com - 20b3974
• butttoucher.com - 0e04590
• teenplanet - acc7f87
• xhamster (images) - 597a7b3
• 8muses - 14afa96
• seenive - 3deddf3
• vinebox - 8bfae56
• photobucket - 2875ee8
• flickr - bf100f2
• minus - 55aef94
• anonib - 0ff9fa6
• fapproved - b5e10c0
• gifyo - 39bca3b
• vine - a0356c8
• supertangas - 81bbcab
• drawcrowd - 339aa89
• modelmayhem - 863f722
• smutty.com - e9bee91
• imagestash.org - 0073de1
• imgbox.com - 13d13b2
• mediacru.sh - 574b03e
• nfsfw - 5d31935
• imagebam - c695733
• pornhub - 6d55e2c
• imagevenue - 838ad2a
• hentai-foundry - ca4ed62
• fuskator - 53df6ca
• datw.in - 5afbc1c
• furaffinity - 9e40e55
• 500px - baab27d
• taptastic - a95bfa7
• cheeby - 562f224
• bcfakes - aa296ef

Pending sites + examples (NSFW):

• getgonewild - http://www.getgonewild.com/profile/rollmeup
• blogspot - http://marcherryfashion.blogspot.com/
• pixiv - http://www.pixiv.net/member.php?id=2136123
• facebook - https://www.facebook.com/EmanueleFerrariPhotographer/photos_stream
• pinterest - http://www.pinterest.com/
• lookbook - http://lookbook.nu/mayowo
• 4shared - http://www.4shared.com/folder/pTUpT6wG/Heather.html
• behance - https://www.behance.net/HS-nj
• dribbble - https://dribbble.com/studioMUTI
• fetlife - https://fetlife.com/users/2089769
• diverxity - http://www.diverxity.com/users/953
• artician - http://zionenciel.artician.com/
• studio-g - http://studio-g.tistory.com/535
• fg-site - http://www.fg-site.net/members/umeume/
• gelbooru - http://gelbooru.com/index.php?page=post&s=list&tags=futa
• thefappening.sexy - http://thefappening.sexy/albums/index.php?/category/126
• ipernity - http://www.ipernity.com/doc/kimcums/album/676241
• punterplanet - http://guide.punterplanet.com/escorts/featured/frontpage/amber-green_200.html
• rule34 - rule34.paheal.net
• photoblog.pl - http://www.photoblog.pl/colordust/170020651/typical-girl.html
• megamodels.pl - http://www.megamodels.pl/a-ya
• maxmodels.pl - http://www.maxmodels.pl/modelka-agnes-z-garbledville.html
• flog.pl - http://www.flog.pl/szukaj/akt

Enhancement requests:

• flickr - login support (see #32)
• vk - Video ripper rename files
• twitter - Search ripping
• gifyo - Don't skip some images
• *chan rippers - Handle external links (vocaroo), store comments
• tumblr - Rip by date (/archive/year/month)

After a finished rip, display in short how many files were downloaded (like on the site) and how many errored out.

This is not a bug, just a thing I think should be corrected:

I was downloading this:
goneabitwild.imgur.com
but this happened:

2014-04-29 16:48:37,953 22945 [pool-1-thread-2] INFO ripper.DownloadFileThread - Downloading file: http://i.imgur.com/VpStI.jpg
2014-04-29 16:48:38,347 23339 [pool-1-thread-2] INFO ripper.DownloadFileThread - [+] Saved http://i.imgur.com/VpStI.jpg as ./rips/imgur_goneabitwild/8OUfg/002_VpStI.jpg
2014-04-29 16:48:47,004 31996 [Thread-6] INFO ripper.AbstractRipper - [!] Skipping http://i.imgur.com/VpStI.jpg -- already attempted: ./rips/imgur_goneabitwild/KTOKb/003_VpStI.jpg

I don't think VpStI.jpg should be skipped in /a/KTOKb

thanks

RipMe seems to slow down and become completely unresponsive when:

• downloading galleries larger than 15,000 images
• using Re-Rip all on a number of galleries that total more than 79,850 images.

It seems once ripme.log reaches a filesize of 20MB, it stops updating the log, and soon after, stops responding.

Keep in mind, I've only tested this with Tumblr, because I've only been able to find galleries of that size there.

Clipboard autorip doesn't work if ripper is already ripping an album.

Rip Queue could be another tab (like "History") showing upcoming downloads.

UI is already pretty busy, though..

Best part of the ripper site was seeing what other people were ripping.

That's gone in the stand-alone client-side version.

Possibilities:

• RipMe tells a server what albums it's downloading; provides URLs to 4 sample images to be thumbnailed.
• Server provides API to get most-recently requested albums and their thumbnails.
• User can decide to re-rip the album to their computer to view the rest.

Not sure how this would fit into the existing UI... seems like a separate window is appropriate.

It would be nice to have the option to add a string of text to either the beginning or end of the file rip folder (top help with organizing/labeling/tagging)

u/micah376

I'm having trouble ripping imgur albums with a subdomain format, for instance: http://goneabitwild.imgur.com/
This issue doesn't occur with imgur albums with the format imgur.com/a/#####

Not sure if there is a problem on my end, but I've confirmed the issue with multiple examples of both types of albums. In case its a bug I thought it was worth reporting.

Old imgur urls were formatted like so.. http://dayah.imgur.com/kapow
Clicking through manually, the new url is http://imgur.com/a/gKA9V

This URL is not supported.
https://fallinloveyoulose.imgur.com/all

URL regex is too strict.

Please add support for the /gallery/ links.

http://www.imagefap.com/gallery/4753165
http://www.imagefap.com/pictures/4753165/exilevilify---reddit-gonewild%21

2014-04-21 02:28:03,936 562 [Thread-5] ERROR ui.MainWindow - Could not find ripper for URL http://www.reddit.com/user/redhotsillyfecker
2014-04-21 02:28:04,778 1404 [AWT-EventQueue-0] INFO ui.UpdateUtils - Running latest version: 1.0.21

Was working in 19.

$ java -jar ripme.jar
Initialized ripme v1.0.38
Loaded /tmp/rip.properties
Running latest version: 1.0.38
http://motherless.com/G4AFA9DF
Could not find ripper for URL http://motherless.com/G4AFA9DF

Usernames with - and _ aren't ripped properly. .

Likely regex/replacement is causing the issue, or the filesystem saves (?)

If you try to rip a user that is not found on gonewild.rarchives it wont rip anything. Just a blank folder. Why wouldn't it rip like the old web based and pull from the profile? Or am I missing something

2014-04-23 12:58:01,512 5901 [Thread-2] ERROR rippers.GonewildRipper  - Error while retrieving user posts:Traceback (most recent call last):
  File "api.cgi", line 213, in <module>
    print dumps(main(), indent=2)
  File "api.cgi", line 29, in main
    elif method == 'get_user':  return get_user(keys)
  File "api.cgi", line 72, in get_user
    count   = int(keys.get('count', 10))
  File "/home/rarchivers/gonewild.rarchives.com/py/Queries.py", line 307, in get_user_posts
    userid = db.select_one('id', 'users', 'username = ?', [user])
  File "/home/rarchivers/gonewild.rarchives.com/py/DB.py", line 213, in select_one
    return one[0]
TypeError: 'NoneType' object is unsubscriptable

2014-04-23 12:58:01,513 5902 [Thread-2] INFO  ripper.AbstractRipper  -    Rip completed!

u , good sir, Rock!!!

After auto-updating 1.0.11 to 1.0.12 the executable file did not rename itself from ripme.jar.update, had to rename it manually.

Sometimes a page doesn't load, or takes a while. Retry logic when this happens could be useful.

Caveats:

• Each ripper has it's own unique settings for retrieving the page (Jsoup's get() or execute(), cookies, referrers, useragents)
• All existing rippers will need to be changed to use the generic getter (and tested)

"the program does not" download images from www.8muses.com when I put the link and press you just rip button stays so.

Link to rip error: http://www.8muses.com/index/category/4323-grumpy_old_man_jefferson_2

I keep getting a "No Protocol" error when trying to use the eHentai ripper

2014-05-13 16:07:14,355 44573 [Thread-8] INFO  ripper.AbstractRipper  - Deleting empty directory /home/Random/rips/e-hentai_Artist_Coco
2014-05-13 16:07:14,968 45186 [AWT-EventQueue-0] INFO  ripper.AbstractRipper  - [+] Creating directory: ./rips/e-hentai_Artist_Coco
2014-05-13 16:07:14,968 45186 [AWT-EventQueue-0] DEBUG ripper.AbstractRipper  - Set working directory to: /home/Random/rips/e-hentai_Artist_Coco
2014-05-13 16:07:14,968 45186 [AWT-EventQueue-0] DEBUG ripper.DownloadThreadPool  - Initializing Main thread pool with 10 threads
2014-05-13 16:07:14,969 45187 [Thread-9] INFO  rippers.EHentaiRipper  -     Retrieving http://g.e-hentai.org/g/######/##########/
2014-05-13 16:07:15,097 45315 [Thread-9] ERROR ripper.AbstractRipper  - Got exception while running ripper:
java.net.MalformedURLException: no protocol: 
    at java.net.URL.<init>(Unknown Source)
    at java.net.URL.<init>(Unknown Source)
    at java.net.URL.<init>(Unknown Source)
    at com.rarchives.ripme.ripper.rippers.EHentaiRipper.rip(EHentaiRipper.java:96)
    at com.rarchives.ripme.ripper.AbstractRipper.run(AbstractRipper.java:290)
    at java.lang.Thread.run(Unknown Source)
2014-05-13 16:07:15,097 45315 [Thread-9] INFO  ripper.AbstractRipper  -    Rip completed!
2014-05-13 16:07:15,097 45315 [Thread-9] INFO  ripper.AbstractRipper  - Deleting empty directory /home/Random/rips/e-hentai_Artist_Coco

Flickr groups not working. Here is a group I wanted to rip
https://www.flickr.com/groups/absolute_bikini/

Hello,

The homepage says that Flickr is one of the supported sites for the ripme program. Some pages are still unable to be ripped. For instance I tried ripping three types of flickr pages and none seem to work.

Tagged Flickr page: https://www.flickr.com/photos/8189261@N05/tags/raych/
Album/Sets Flickr page: https://www.flickr.com/photos/8189261@N05/sets/72157628866931619/with/7052912871/
User Flickr page: https://www.flickr.com/photos/8189261@N05

Flickr pages with numbers like 8189261@N05 instead of usernames don't work at all. Only flickr pages with names in the url such as https://www.flickr.com/photos/constaza/ work. Unfortunately, the program doesn't rip albums or tagged pics in usernames though such as https://www.flickr.com/photos/katj2day/sets/72157638291120306/ -or- https://www.flickr.com/photos/katj2day/tags/sexy/. This can be troublesome if a user has thousands of pics and you only want to get an album or tagged pics.

If it's possible to add numbered users like 8189261@N05, album/set rip capability, and tags rip capability, that'd be fantastic.

Irregardless of all this, I love the program so far and look forward to it getting better everyday. Appreciate all the effort.

Watch system clipboard for when URLs are copied, start ripping automatically.

Uses current working directory.

Also would be nice if the ripper stored config and history files in the /rips/ directory!

https://twitter.com/lila_mfc

log
http://cryptb.in/xe#6e2b1359fb3b3aa4584b6c09dc6be0ab

thanks

Example: Downloading this URL http://www.reddit.com/user/DecoDamsel/
returns a lot of placeholder files http://i.imgur.com/removed.png
would be great to maybe filter them out.

Nice app. However it seems it can rip only the 1st page of a multipage gallery in Deviant art.

Keep up the good work.

Samples:

• http://fiddle.se/
• http://realbarbielifts.com/