Rate Exceeded about acd_cli HOT 82 OPEN

@yadayada

Thank you for all your hard work with acd_cli, been a big fan/user for a long time! 72+TBs.
Had to make a github account, as I don't socialize...

Anyway, I just re-created a new oauth.json token with https://drivesink.appspot.com/config
And pasted it ~/.cache/acd_cli/

It's sloppy but it's working fine for the past couple days 👍

-lurker
d3ath

from acd_cli.

As a small update regarding ACD, it might be time to actually leave the service completely

Ref: https://www.amazon.com/b?node=16591160011

from acd_cli.

Anyone have acd_cli working presently?

I find the timing of banning third party apps right before announcing the end of their unlimited storage plan way too suspicious. If they don't reinstate the third party apps until the unlimited plans expire, I think we should all inform our state Attorneys General that Amazon is engaging in anticompetitive consumer lock-in.

from acd_cli.

@AlexCzar I can neither confirm nor deny that the old/new security profile was banned temporarily or permanently, as I have not received any kind of notification or response from the Amazon Drive team.

I have applied to bump the new Appspot app to production limits, but I'm fairly certain that won't happen. It uses a spare security profile of mine.

You might want to give @chrisgavin's fork a go, see https://github.com/chrisgavin/cheeky_acd_cli.

BTW, I cancelled my renewal:

from acd_cli.

from acd_cli.

for me, acd_cli sync give this:

acdcli.api.common.RequestError: RequestError: 429, [acd_api] no body received.

and Expandrive gives an empty folder.

Amazon is messing again with 3rd party tools

from acd_cli.

So tired of these cloud storage bait-and-switch schemes. First Microsoft, then this.

from acd_cli.

Gentlemen,
Today I got the same problem while trying to connect to Amazon Drive.
Acdcli returns the error:
[Acdcli.api.client] - Error getting endpoint data. Response: {"message": "Rate exceeded"}

I emailed Amazon, where I quickly responded.
I wondered how to use Amazon Drive on Linux.
The sad news is that Amazon Drive supports only connection via web browser.
Acdcli and rclone client apps are not supported. That's why we're not getting access to Amazon Drive.

Here is the answer portion of the email:

Hello,

I understand your concern about using third party clients and apps to access Amazon Drive on Linux.
Unfortunately, third party applications such as, acd_cli and Rclone client access has been revoked and they no longer supports Amazon Drive access. I'm sorry for any inconvenience this might have caused to you.
I'd suggest you to please access Amazon Drive using website on Linux. Here's the link:

https://www.amazon.com/clouddrive

from acd_cli.

yeah, I used multcloud - took it about 3 weeks to move over 8TB - but it got everything done for $8

from acd_cli.

Ditto:

Traceback (most recent call last):
  File "/usr/local/bin/acd_cli", line 9, in <module>
    load_entry_point('acdcli==0.3.2', 'console_scripts', 'acd_cli')()
  File "/usr/local/bin/acd_cli.py", line 1688, in main
    ret = args.func(args)
  File "/usr/local/bin/acd_cli.py", line 849, in usage_action
    r = acd_client.get_account_usage()
  File "/usr/local/lib/python3.5/dist-packages/acdcli/api/account.py", line 67, in get_account_usage
    raise RequestError(r.status_code, r.text)
acdcli.api.common.RequestError: RequestError: 429, {"message":"Rate exceeded"}

from acd_cli.

I figured I'd try to enable my own security profile as outlined here, but then I started getting invalid_scope errors when trying to get a token under my new profile.

And now this page has the following message:

The Amazon Drive API and SDKs are currently closed to new developers.
Thank you for your interest. We received an overwhelming number of API invitation requests with many innovative integration scenarios. We have ended the invitation period to focus on enabling new customer experiences with current developers.

I'm unsure if this is related, or how recent that message is, but it's crap timing. :-/

from acd_cli.

Same here, same stacktrace as @nemymongus.

from acd_cli.

It would probably be safe to assume that this is just another Amazon Cloud Drive quota we blew over (#550). I would wait until things settle, I doubt this is the end again, unless @yadayada went behind Amazon's back and is using a different CLIENT_ID and never had their ban lifted in the first place.

This is likely just Amazon's throttling as described in #416

from acd_cli.

Still get 429 even switching between multiple authorized apps. Perhaps something wrong on the backend.

In the response headers:
x-amzn-ErrorType: TooManyRequests:http://internal.amazon.com/coral/com.amazon.clouddrive.service/

Body:
{"code":null,"info":null,"logref":null,"message":"Rate exceeded"}

from acd_cli.

Yep. I'm getting a 429 with a different app too.

from acd_cli.

Works fine with Advanced Users (Security Profile) auth method.
Yes, it means you have to register your own app on https://developer.amazon.com

from acd_cli.

@karbentur, registering a security profile won't work anymore because Amazon have to whitelist it and they aren't accepting any new apps. I'm also not convinced it would help because the security profiles for several apps seem not to be working.

from acd_cli.

@chrisgavin, ok, now I see.
Anyway, security profiles method still works fine.

from acd_cli.

@karbentur how can you say it works fine? I have a security but I cant generate a new oauth key? I the page to generate them is gone?

from acd_cli.

I have the same problem:
[CRITICAL] [acdcli.api.client] - Error getting endpoint data. Response: {"message":"Rate exceeded"}
Traceback (most recent call last):
File "/usr/bin/acd_cli", line 11, in
load_entry_point('acdcli==0.3.2', 'console_scripts', 'acd_cli')()
File "/usr/bin/acd_cli.py", line 1646, in main
acd_client = client.ACDClient(CACHE_PATH, SETTINGS_PATH)
File "/usr/lib/python3.5/site-packages/acdcli/api/client.py", line 46, in init
self._load_endpoints()
File "/usr/lib/python3.5/site-packages/acdcli/api/client.py", line 69, in _load_endpoints
self._endpoint_data = self._get_endpoints()
File "/usr/lib/python3.5/site-packages/acdcli/api/client.py", line 80, in _get_endpoints
raise Exception
Exception

from acd_cli.

+1

from acd_cli.

+1

from acd_cli.

Same here

from acd_cli.

+1

from acd_cli.

My Expandrive is also not working.

from acd_cli.

From Expandrive support chat, they are getting error 429 as acd_cli
but it's not an oauth error like last time

from acd_cli.

That's actually a good news in a sense that it could be temporary. As Expandrive is a paid (and an expensive one) service and the developers play by the rules but even they're down.

from acd_cli.

OR, ACD decided to shut down 3rd party tools. Remeber that the 3rd party tools web page has been removed from amazon few days ago

from acd_cli.

@Kedryn Yep that's what I'm concerned about. Guess all we can do is wait.

from acd_cli.

That would be a pretty stupid move on Amazon's account because I haven't seen many people using it as a "backup/restore" only storage. Even if people do then Amazon hasn't made even a remotely useful app or web interface to perfectly utilize stored media. That's enough inconvenience for me to switch to some other cloud storage with a lot less space.

from acd_cli.

Oh dear. Hopefully this is just the support person being incompetent and getting confused over the recent OAuth issues. If they really have banned all third party tools intentionally then ACD is completely useless.

On the plus side, maybe this is the motivation needed to reverse engineer the "official" client.

from acd_cli.

@ricardoclaus I think they are repeating old news to you (I hope). That's one of the the 3 things people seemed to be told by first level support when calling about this last time. Given that other apps are also not working and all this has happened several times before, I would treat this as a likely Amazon problem at least for today.

Or maybe too many people started syncing once acd_cli was reauthorized by Amazon and we have collectively exceeded Amazon's daily quota for acd_cli

from acd_cli.

ODrive is still working, so they haven't banned all the 3rd party clients (yet).

from acd_cli.

from acd_cli.

ODrive is kind of Amazon's buddy. They where there in the first hour and Amazon used their auto-sync feature to promote drive. At the time the official app was just manual and quite unusable. I highly doubt they would ban them that easily.

from acd_cli.

Shameless plug, but there's a discord server based on discussing future of unlimited cloud storage providers & future of acd_cli

I know last time the issues got out of hand, so there's a acd_cli channel there should anyone be interested.

from acd_cli.

The official acd_cli chat is on gitter though, as linked from the main github page: https://gitter.im/yadayada/acd_cli

from acd_cli.

I think what we are about to see is amazon is acquiring or partnering with some of the providers and the rest are dust.

from acd_cli.

Probably this issue is related to: https://www.arqbackup.com/blog/arq-amazon-drive-outage-resolved/

Soooo just Amazon not being the greatest at communicating what's going on i guess. Unless @yadayada maybe had some sort of NDA on him until this issue (at amazon not acd_cli) is resolved?

from acd_cli.

Interesting to see that arq is back... but I'm still seeing Rate exceeded messages on acd_cli.

from acd_cli.

I think lots of apps have been banned for having public client secrets like rclone was. Arq seemed to have been banned mistakenly for this reason but access was then restored.

acd cli has probably been banned for using another app's security profile. I don't think it's coming back :(

from acd_cli.

Same here (using acd_cli 0.3.2, api 0.9.3) - has anybody been able to get round this yet?

17-06-04 15:17:23.009 [CRITICAL] [acdcli.api.client] - Error getting endpoint data. Response: {"message":"Rate exceeded"}
Traceback (most recent call last):
  File "/home/fff/.local/bin/acdcli", line 11, in <module>
    load_entry_point('acdcli==0.3.2', 'console_scripts', 'acdcli')()
  File "/home/fff/.local/bin/acd_cli.py", line 1657, in main
    acd_client = client.ACDClient(CACHE_PATH, SETTINGS_PATH)
  File "/home/fff/.local/lib/python3.5/site-packages/acdcli/api/client.py", line 46, in __init__
    self._load_endpoints()
  File "/home/fff/.local/lib/python3.5/site-packages/acdcli/api/client.py", line 63, in _load_endpoints
    self._endpoint_data = self._get_endpoints()
  File "/home/fff/.local/lib/python3.5/site-packages/acdcli/api/client.py", line 80, in _get_endpoints
    raise Exception
Exception

from acd_cli.

I have the same problem. I too wrote to Amz support for confirmation. If it is confirmed that acd won't work anymore, I'll have to switch to a different service, as ACD is completely unusable for me without acd_cli. Will probably switch to SpiderOak

from acd_cli.

The thing with SoiderOak is they only provide plans upto 5TB and is costly compared to ACD.

Does anyone have any suggestions other than GoogleDrive as an alternative. I'm looking to store about 20TB total.

from acd_cli.

@lloydie2009 they've updated their plans, they have unlimited now and others > 5TB, yes, it's costly, unlimited is $150 per year, I think. But ACD is indeed absolutely unusable for me without acd_cli.
@yadayada can you please confirm that this outage is permanent and we won't see acd_cli working again due to the latest Amazon policy change?

from acd_cli.

@yadayada Which Cloud Storage are you switching to next?

from acd_cli.

@d3athh Thank you very much !
It works very well ;)

from acd_cli.

RIP

from acd_cli.

Wow! acd_cli works iwth drivesink. Now I'm totally confused, I've already canceled my subscription to ACD it will expire in August. But with this method working I just don't know... will it still be working in 3 months? in 6?
@yadayada in light of this will you still be abandoning acd_cli?

from acd_cli.

I don't mean to hijack this, but I gave up on ACD_CLI and tried a commercial tool I already had, which may be of interest (I have no connection except as a customer). Goodsync is primarily a windows/mac product and commercial, not open source (about $30 for a license). They also offer a free Linux command line version, and they support ACD with both. The catch on the command line version is that it is poorly documented if used alone, the simple and recommended way is to build a job with windows or mac, then export the job and run it on linux (possibly editing the XML-like definition file). This is more arcane than it may sound as the file format is ill defined as well as ill documented, but the product itself is quite solid. The sync aspect is both 2 way sync and also varieties of one-way sync, e.g. backup. It will also do delete-on-copy one-way sync, e.g. move/archive type. I spent a day or so banging at the syntax and managed to convert my ACD_CLI scripts to use Goodsync's linux server instead. So far, and a few gig and maybe 3000 files later, it is uploading without error (well, without substantial error, it gets a few errors, but properly saves the file to retry). So for $30 there's an alternative of sorts that being commercial may not (MAY not) end up being killed by Amazon.

I liked and used ACD_CLI a long time, sorry to see it not going well, and hope it comes back. I just couldn't wait much longer hoping.

from acd_cli.

@Linwood-F were you able to get it to work on a FUSE mount by any chance? I use my ACD instance for both backups and supplementary storage between multiple servers, so I need both command line and mountable.

from acd_cli.

I did not try anything else; I used it just to archive jpg files off from a security system to ACD, so I was copying a whole folder at a time with ACD_CLI, which worked nicely with a sync program, I just had it do a 1-way sync of a folder (a quick call to SED to replace the folder names in the job file let me change which folders for which runs). Sorry, no help there. And this alternative does take some effort to get going; their software is nice in a windows/mac GUI, but the command line is pretty arcane -- it's well documented in one sense, but does not give you the syntax that the various cloud providers need, so getting syntax right is a challenge, though it's pretty easy to overcome by doing it in the gui then exporting the job and editing to change windows to linux paths.

from acd_cli.

@Linwood-F thanks. They have 30-Day trial, so I'll give that a go this weekend. Thanks for the Windows tip, will try yoinking the syntax from my Windows machine and pouring over to the Linux. See what kinda luck I have. Thanks again

from acd_cli.

Looks like goodsync might work well enough under wine to get the job xml at least, https://appdb.winehq.org/objectManager.php?sClass=application&iId=11981

from acd_cli.

Export the job though. You can also take the whole file (if you can figure out which one, I still am not sure) from windows, but if you export it, the file contains the account information as well as folders, connections, etc. There's a lot of repeated junk in there, but if you set up a job to move 1 folder, with one-way sync from windows to ACD, and set any other options you want (like move vs copy, threads, probably turn off propagate deletions) then name.tix file will be all you should need, then run the program on linux wiith

gsync sync jobname /job-file=jobfilename.tix

it should work, though first edit to change windows paths to linux paths. The online manual documents most of the contents, though gives short shrift to the syntax, so a working file first is useful, the prune and change and try as you go. Do take care as being a (possibly) 2 way sync program it absolutely can wipe out the "from" side if you are not careful.

from acd_cli.

@asabla WOW! just wow... Google has gone in with unlimited storage for education and work and Amazon is now trying to get out. I hope this decision doesn't influence what Google does with their unlimited storage plans. I'm trying to sync my stuff out right now and as soon as it's done I'm asking for a full refund.

from acd_cli.

I was already removing my files from ACD, while i saw this change. I use amazon.it that still state the original 70€ per year unlimited, but i terminated the service and had a chat with support to ask the full refund. The guy did tell me that the amazon.it is not going to change it's plans, but i cant really believe that 15TB for .com is going to cost 900$ a year, while for amazon.it 70€.

from acd_cli.

https://techcrunch.com/2017/06/08/amazon-ends-its-unlimited-cloud-storage-plan/?ncid=rss

from acd_cli.

Thanks @Joshua1337 - peace out Amazon, another option for cloud storage bites the dust. For that price-tag, I'll buy my own storage space and host it locally. F-it.

from acd_cli.

@bdeluca lol - that is fantastic

from acd_cli.

upload encrypted files with image format extensions... they're "pictures" lol

from acd_cli.

Hey guys, just in the FWIW department, all that kind of action will do is spoil it for people how need the photo storage when it forces them to back away from all-you-can-eat photos as well; it's not really going to teach Amazon a lesson (not even sure Amazon deserves a lesson; while they were hardly communicative, the ACD_CLI issue seemed to stem from a legitimate security issue, it's not as though they just wacked our use for spite). I'm not the morality police or anything, but make sure the people you punish are not people like yourselves.

Update midnight UTC 6/9/2017: Now ACD isn't working even from the web interface much less API, so I wonder if upon announcement a ton of people started downloading (or attacking). Interesting it follows substantial outage of amazon's store yesterday afternoon.

from acd_cli.

from acd_cli.

Moving from Amazon to Hubic - 10TB for ~$55/year

from acd_cli.

@blackyangell Why you not using GDrive, it's $120/year, but it's unlimited and you have 10Tb transfer rate per day.

from acd_cli.

GDrive - Google Drive?
I see $99.99 / month per 10TB, it's $1200/year.
And there is no unlimited plan

from acd_cli.

Has anybody tried connecting GSuite with Plex Cloud and in return triggered a hash check?

from acd_cli.

@blackyangell Try searching GSuite.

from acd_cli.

@blackyangell You need to up your game on reading forum and other places, GSuite, offer unlimited storage for 1 user (which even they said 5 user) for $10/month.

@AiMAnsarie
Mine is connected to Plex Cloud and so far no problem :)

from acd_cli.

@dany20mh, what does it mean - "Use Drive to keep all your work files in one secure place with unlimited cloud-based file storage (accounts with fewer than 5 users get 1TB per user)"

I can't get it - Unlimited or 1TB?

And can I use it as cloud storage, without synchronization with my computer like Google Drive ?

from acd_cli.

@blackyangell
I just said that, that's their requirement for having unlimited storage, but they not enforcing it, so you can sign up with one user and have unlimited storage.

And you can use rclone, plexdrive, ocamlfuse, node-gdrive and other app to connect to the storage and have it sit in front of you without taking any space.

from acd_cli.

@blackyangell yes you can use it without synchronization.

what you see there is a fuse mount of google drive inside a linux vm inside my home server. totally worth it.

thats what i use: https://github.com/mitcdh/docker-google-drive-ocamlfuse

thats what my docker-compose file looks like:

version: '3.3'
services:
 gdrive:
   container_name: gz-drive
   image: mitcdh/google-drive-ocamlfuse
   cap_add:
     - mknod
     - sys_admin
   volumes:
     - /mnt/gdrive:/mnt/gdrive:shared
     - ./data:/config/.gdfuse/default
   devices:
     - /dev/fuse:/dev/fuse
   environment:
     - CLIENT_ID=asdfg
     - CLIENT_SECRET=foobar
     - VERIFICATION_CODE=om nom nom

there are other alternatives aswell. you might just want to google a bit.
so far it seems like google drive is far more usable than amazon drive.

now i still have to figure out how to move my 5 tb from amazon over to google...
i did not came into this issue randomly.. i have the same issue as you guys 😭

from acd_cli.

thank you very much. works like a charm.

from acd_cli.

Just started migrating data ACD->GDrive with multcloud. Multcloud shows 10TB of free space on GDrive. Isn't it supposed to be unlimited? I did not hit that point yet, but I was wondering if anyone could confirm that GDrive is actually unlimited.

from acd_cli.

yea GDrive is unlimited when you use some space your complete space should go up. Like you use 1TB it will show you 11TB Complete and 10TB Free.

from acd_cli.

nice. thanks for the info.

from acd_cli.

Grumble. I'd really enjoyed Amazon Cloud Drive thanks to acd_cli, but I also just canceled my subscription because of this. Here's my full tale of woe.

I'd actually been struggling for the past couple months with a technical issue because when Amazon announced the unlimited plan was going away, I wanted to get my storage use below 1 TB to save money. I deleted everything I could see, and yet their interface still claimed I was using 703 TB. I opened a ticket and despite repeated proddings they did nothing to address it for 2 months. Then when they redesigned their web interface, I could suddenly see the 703 GB worth of data under "Recent Files" even though none of it was under "All Files". So, still buggy, but at least I could delete it all and start from zero. Then when I tried to restart from zero I found that acd_cli was no longer permitted to work. Like everyone else here, I find the web/desktop clients completely unusable, so, we're done here.

After being annoyed at the limitations and pricing of alternatives, I've decided to just buy a 3 TB Toshiba bus-powered USB hard drive, back up my stuff to that, and carry it to the office periodically for safe offsite storage. If I miss being able to access my files over the Internet, perhaps I'll hook it up to a spare laptop there and open up an SSH tunnel to it. The cloud is cute, but I'm getting weary of not being in control. Screw this.

from acd_cli.

Hi guys. Little late for the party.. Anyone have a nice alternative to [ACD] -> [Amazon Drive]? Just want to make sure before I move to other "unlimited" storage.

from acd_cli.

They ruined their own product. Silver lining: if you write them how bad move they made they may gift you $10 Amazon credit, at least they did in my case, as if $10 can fix discontinuation of such a service.

from acd_cli.

@d3athh is this still working for you? I tried it and it expires in like an hour or something. I am just trying to get this to work until my subscription expires and then I will find an alternative to Amazon Cloud.

from acd_cli.

Is this an acd_cli is dead thread? I have been automating my syncs with acd for a while and just realized the backups haven't been working. Should I move on?

17-12-18 17:39:40.659 [CRITICAL] [acdcli.api.client] - Error getting endpoint data. Response: {"message":"Rate exceeded"}

from acd_cli.

@2marshall yup you should. Amazon killed it.

from acd_cli.