akdeniz / google-play-crawler Goto Github PK
View Code? Open in Web Editor NEWPlay with Google Play API :)
License: Other
Play with Google Play API :)
License: Other
HI!
Firstly, thank you for a great lib!
After update a version I have the error
I work with library in jruby
After import com.akdeniz.googleplaycrawler.GooglePlayAPI
I have error in log
LoadError: load error: ./lib/google_play -- java.lang.VerifyError: class com.akdeniz.googleplaycrawler.GooglePlay$ListResponse overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
require at org/jruby/RubyKernel.java:1027
(root) at script/google_play.rb:8
May be do you know how it resolve?
I would like to know if i can get what kind of architecture is supported by apk?.
is x86 supported?.
is optimized for tablets?.
I continuosly get empty apps Title(Name)
DetailsResponse detailsResponse = service.details(packageName);
DocV2 docV2 = detailsResponse.getDocV2();
DocumentDetails details = docV2.getDetails();
AppDetails appDetails = details.getAppDetails();
appDetails.getTitle() // is always empty..
What is wrong with my code ? Thanks.
It perfectly works without this method call. What is the purpose of this method ? Thanks.
We've been using and reading your googleplay java api, and it works perfectly well in downloading an app from the play store. We were trying to go a step further and see if those downloads could also lead to an increase in install numbers on the play store. But that didn't happen, and in order to make that happen:
We observed the following requests when Gplay market downloads an app:
Apart from these, there was another request, the following, which was being sent at random times. This contained a Base64 encoded string which, as far as we could apprehend had Authtoken, logging id(constant for a device), device_id(gsf_id for the device), and a string that had list of package names installed which we think are protobuffed and we couldn't figure out.
https://android.clients.google.com/market/api/ApiRequest
Here is the trace for one such request :
POST https://android.clients.google.com/market/api/ApiRequest HTTP/1.1
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
User-Agent: Android-Market/2
X-Public-Android-Id: 6532b095d0193d17
Content-Length: 7183
Host: android.clients.google.com
Connection: Keep-Alive
version=2&request=CuMCCosCRFFBQUFMb0FBQURjQnUxX1JmM3pfYzhhMnpLbmhMamh3UmVpVUxiUDl1YTNheEEwalVyWTBwSW9ieTZNTFNmdlc1MUNnTkhReGN2cTBVek1tZkI0M1R5UnFCSWtwLXZ4UDhRYkFnLTVXTFhYZTVDS0I4U2xRa1Q2QXlPVFQyQlpaTk4zRWJlZTB6SGdlODlYX1JqamtSWkRJMklROFJMSEdQTi10WmNwMmhGZGZVWVY5Wkl2UzIzYjA4RHR4Z09PWXdvZkFCTjJRYU53UW9oN2NsdFlVQkVuTktLMlVLT0RTTGVVZ3E4V2kxYXBnZEFlTzIydElubmgyRGNWRldpaGVGVXlCZG94cnFnEAEYlbuhJiIQMzdhYWVjZDk0YmExNWVjNSoLdmJveDg2dHA6MTYyAmVuOgJVU0IASgBSAFoAYgphbS11bmtub3duahEtNjg2ZTVjZWMyZmZjMjM3MhNKkicTGiR2Mjpjb20ubGlvbWlpLndhdGVyLnB1enpsZS5sb2dpYzoxOjMgAiizvp3Spyg6HWNvbS5saW9taWkud2F0ZXIucHV6emxlLmxvZ2ljQAMUExoddjI6Y29tLnRlc3RhcHAuaGFzb2ZmZXJzNjoxOjMgAiizvp3Spyg6FmNvbS50ZXN0YXBwLmhhc29mZmVyczZAAxQTGh52MjptZS5zb3VuZHdhdmUuc291bmR3YXZlOjE6OTAgAiizvp3Spyg6Fm1lLnNvdW5kd2F2ZS5zb3VuZHdhdmVAWhRTWgdhbmRyb2lkYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWhRjb20uYW5kcm9WTS52bWNvbmZpZ2ABahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oZY29tLmFuZHJvaWQuYmFja3VwY29uZmlybWAQahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oVY29tLmFuZHJvaWQuYmx1ZXRvb3RoYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWhNjb20uYW5kcm9pZC5icm93c2VyYBBqG1llMDNmb1hUaHFqZjdtdUdTOWhiQ19xbHI0RVRTWhdjb20uYW5kcm9pZC5jYWxjdWxhdG9yMmAQahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oUY29tLmFuZHJvaWQuY2FsZW5kYXJgEGobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaEmNvbS5hbmRyb2lkLmNhbWVyYWABahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oZY29tLmFuZHJvaWQuY2VydGluc3RhbGxlcmAQahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oUY29tLmFuZHJvaWQuY29udGFjdHNgEGobV3phTV95MmlhR21XdkpYcXdaRHFwUFZqRC1VVFNaGGNvbS5hbmRyb2lkLmRlZmNvbnRhaW5lcmAQahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oVY29tLmFuZHJvaWQuZGVza2Nsb2NrYMsBahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oRY29tLmFuZHJvaWQuZW1haWxgkIMZahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oUY29tLmFuZHJvaWQuZXhjaGFuZ2VgoMIeahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oTY29tLmFuZHJvaWQuZ2FsYXh5NGABahtXemFNX3kyaWFHbVd2Slhxd1pEcXBQVmpELVVUU1oTY29tLmFuZHJvaWQuZ2FsbGVyeWAQaht0NTMwcUM2UXRYNm5aU1dyY0Rlckk0cEM5ZE1UU1oWY29tLmFuZHJvaWQuaHRtbHZpZXdlcmAQahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oYY29tLmFuZHJvaWQuaW5wdXRkZXZpY2VzYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWh1jb20uYW5kcm9pZC5pbnB1dG1ldGhvZC5sYXRpbmAQahtXemFNX3kyaWFHbVd2Slhxd1pEcXBQVmpELVVUU1oeY29tLmFuZHJvaWQuaW5wdXRtZXRob2QucGlueWluYBBqG1d6YU1feTJpYUdtV3ZKWHF3WkRxcFBWakQtVVRTWhRjb20uYW5kcm9pZC5rZXljaGFpbmAQahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oUY29tLmFuZHJvaWQubGF1bmNoZXJgEGobV3phTV95MmlhR21XdkpYcXdaRHFwUFZqRC1VVFNaJGNvbS5hbmRyb2lkLmxpdmV3YWxscGFwZXIubWljcm9iZXNnbGAPahtuRUZ5bEhHeWtQeUxJQXZQWXRLVWQtc1lHdWtUU1oWY29tLmFuZHJvaWQubWFnaWNzbW9rZWAQahtXemFNX3kyaWFHbVd2Slhxd1pEcXBQVmpELVVUU1oPY29tLmFuZHJvaWQubW1zYBBqG1llMDNmb1hUaHFqZjdtdUdTOWhiQ19xbHI0RVRTWhFjb20uYW5kcm9pZC5tdXNpY2AQahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oTY29tLmFuZHJvaWQubXVzaWNmeGCgUWobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaFGNvbS5hbmRyb2lkLm11c2ljdmlzYBBqG1d6YU1feTJpYUdtV3ZKWHF3WkRxcFBWakQtVVRTWhZjb20uYW5kcm9pZC5ub2lzZWZpZWxkYAFqG1d6YU1feTJpYUdtV3ZKWHF3WkRxcFBWakQtVVRTWhxjb20uYW5kcm9pZC5wYWNrYWdlaW5zdGFsbGVyYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWhVjb20uYW5kcm9pZC5waGFzZWJlYW1gAWobV3phTV95MmlhR21XdkpYcXdaRHFwUFZqRC1VVFNaEWNvbS5hbmRyb2lkLnBob25lYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWiJjb20uYW5kcm9pZC5wcm92aWRlcnMuYXBwbGljYXRpb25zYBBqG1d6YU1feTJpYUdtV3ZKWHF3WkRxcFBWakQtVVRTWh5jb20uYW5kcm9pZC5wcm92aWRlcnMuY2FsZW5kYXJgEGobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaHmNvbS5hbmRyb2lkLnByb3ZpZGVycy5jb250YWN0c2AQahtXemFNX3kyaWFHbVd2Slhxd1pEcXBQVmpELVVUU1ofY29tLmFuZHJvaWQucHJvdmlkZXJzLmRvd25sb2Fkc2AQaht0NTMwcUM2UXRYNm5aU1dyY0Rlckk0cEM5ZE1UU1oiY29tLmFuZHJvaWQucHJvdmlkZXJzLmRvd25sb2Fkcy51aWAQaht0NTMwcUM2UXRYNm5aU1dyY0Rlckk0cEM5ZE1UU1oZY29tLmFuZHJvaWQucHJvdmlkZXJzLmRybWAQaht0NTMwcUM2UXRYNm5aU1dyY0Rlckk0cEM5ZE1UU1obY29tLmFuZHJvaWQucHJvdmlkZXJzLm1lZGlhYP0Daht0NTMwcUM2UXRYNm5aU1dyY0Rlckk0cEM5ZE1UU1oeY29tLmFuZHJvaWQucHJvdmlkZXJzLnNldHRpbmdzYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWh9jb20uYW5kcm9pZC5wcm92aWRlcnMudGVsZXBob255YBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWiRjb20uYW5kcm9pZC5wcm92aWRlcnMudXNlcmRpY3Rpb25hcnlgEGobV3phTV95MmlhR21XdkpYcXdaRHFwUFZqRC1VVFNaFWNvbS5hbmRyb2lkLnByb3Zpc2lvbmAQahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oUY29tLmFuZHJvaWQuc2V0dGluZ3NgEGobSnhsdU9HdUhYbmF0OXdEbjZvVGt4dTdqUGZvVFNaH2NvbS5hbmRyb2lkLnNoYXJlZHN0b3JhZ2ViYWNrdXBgEGobSnhsdU9HdUhYbmF0OXdEbjZvVGt4dTdqUGZvVFNaE2NvbS5hbmRyb2lkLnNtc3B1c2hgEGobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaGWNvbS5hbmRyb2lkLnNvdW5kcmVjb3JkZXJgEGobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaFGNvbS5hbmRyb2lkLnN5c3RlbXVpYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWhNjb20uYW5kcm9pZC52ZW5kaW5nYJW7oSZqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWhdjb20uYW5kcm9pZC52aWRlb2VkaXRvcmALahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oXY29tLmFuZHJvaWQudm9pY2VkaWFsZXJgEGobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaFmNvbS5hbmRyb2lkLnZwbmRpYWxvZ3NgEGobSnhsdU9HdUhYbmF0OXdEbjZvVGt4dTdqUGZvVFNaFWNvbS5hbmRyb2lkLndhbGxwYXBlcmAQahtXemFNX3kyaWFHbVd2Slhxd1pEcXBQVmpELVVUU1ogY29tLmFuZHJvaWQud2FsbHBhcGVyLmhvbG9zcGlyYWxgEGobV3phTV95MmlhR21XdkpYcXdaRHFwUFZqRC1VVFNaIGNvbS5hbmRyb2lkLndhbGxwYXBlci5saXZlcGlja2VyYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWhtjb20uY3lhbm9nZW5tb2QuZmlsZW1hbmFnZXJgZWobSnhsdU9HdUhYbmF0OXdEbjZvVGt4dTdqUGZvVFNaHWNvbS5leGFtcGxlLmFuZHJvaWQubGl2ZWN1YmVzYBBqG1llMDNmb1hUaHFqZjdtdUdTOWhiQ19xbHI0RVRTWiljb20uZ29vZ2xlLmFuZHJvaWQuYXBwcy5nZW5pZS5nZW5pZXdpZGdldGCfCmobSkxza3dGNUg0Szc2YUtXS2RtRjUyYllUcGdBVFNaIGNvbS5nb29nbGUuYW5kcm9pZC5hcHBzLnVwbG9hZGVyYMC4AmobT0pHS1JUMEhHWk5VLUxHYThGN0dWaXp0VjRnVFNaGWNvbS5nb29nbGUuYW5kcm9pZC5iYWNrdXBgEGobSnhsdU9HdUhYbmF0OXdEbjZvVGt4dTdqUGZvVFNaG2NvbS5nb29nbGUuYW5kcm9pZC5mZWVkYmFja2AQahtPSkdLUlQwSEdaTlUtTEdhOEY3R1ZpenRWNGdUU1onY29tLmdvb2dsZS5hbmRyb2lkLmdvb2dsZXF1aWNrc2VhcmNoYm94YIC5x19qG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWhZjb20uZ29vZ2xlLmFuZHJvaWQuZ3NmYBBqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWhxjb20uZ29vZ2xlLmFuZHJvaWQuZ3NmLmxvZ2luYBBqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWhtjb20uZ29vZ2xlLmFuZHJvaWQubG9jYXRpb25g1ghqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWiJjb20uZ29vZ2xlLmFuZHJvaWQubWFydmluLnRhbGtiYWNrYERqG20wSk1MU2V0VWFRcU0zNEx0cGtjZHV5a1JHRVRTWiVjb20uZ29vZ2xlLmFuZHJvaWQub25ldGltZWluaXRpYWxpemVyYBBqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWh9jb20uZ29vZ2xlLmFuZHJvaWQucGFydG5lcnNldHVwYBBqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWh5jb20uZ29vZ2xlLmFuZHJvaWQuc2V0dXB3aXphcmRgggFqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWiljb20uZ29vZ2xlLmFuZHJvaWQuc3luY2FkYXB0ZXJzLmJvb2ttYXJrc2AQahtPSkdLUlQwSEdaTlUtTEdhOEY3R1ZpenRWNGdUU1ooY29tLmdvb2dsZS5hbmRyb2lkLnN5bmNhZGFwdGVycy5jYWxlbmRhcmAPahtPSkdLUlQwSEdaTlUtTEdhOEY3R1ZpenRWNGdUU1ooY29tLmdvb2dsZS5hbmRyb2lkLnN5bmNhZGFwdGVycy5jb250YWN0c2AQahtPSkdLUlQwSEdaTlUtTEdhOEY3R1ZpenRWNGdUU1oXY29tLmdvb2dsZS5hbmRyb2lkLnRhbGtgygJqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWhZjb20uZ29vZ2xlLmFuZHJvaWQudHRzYBBqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWh5jb20uZ29vZ2xlLmFuZHJvaWQudm9pY2VzZWFyY2hggJL0AWobT0pHS1JUMEhHWk5VLUxHYThGN0dWaXp0VjRnVFNaF2NvbS5ub3NodWZvdS5hbmRyb2lkLnN1YC9qG1h4RV93c0lLZkp2WktCa2lhaktoa0V0MTc0c1RTWg1jb20uc3ZveC5waWNvYAFqG1llMDNmb1hUaHFqZjdtdUdTOWhiQ19xbHI0RVRTWhRjb20udGYudGhpbmtkcm9pZC5zZ2DnswpqGzl2UVlPS0QyS3ozMTViZVJfXy1iYlZ4OG1aY1RTWhdqcC5jby5vbXJvbnNvZnQub3BlbndubmAQahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUcAQU&
Your code simulates the following 5 requests:
After using your functions for the above requests we tried to simulate the following requests ourselves:
-https://android.clients.google.com/fdfe/log
This request contains a protobuffed string(as per our understanding) which seems to be some kind of a confirmation for downloading the app to the device.
This is how the string looks: = ( 4confirmFreeDownload?doc=com.google.android.apps.maps
-https://android.clients.google.com/market/api/ApiRequest
We tried to construct the Base64 encoded string, and the protobuffed string for apps that are installed on a particular device, but recieved a 400 response
We replicated a request sent by a Device without changes but we got 400 response.
(We could not figure out how to get the logging_id as well.)
We need help in reconstructing the same request, and understanding how the the data being sent in the request is formed so that we can figure out the last three requests and complete the install cycle.
Any help would be highly appreciated since you're the ultimate authority on this. :-)
Hello,
First of all I'd like to say thank you for this awesome library !
I want to work with this stuf in multithreading environment with a few concurrent GooglePlayAPI clients. Is it possible to setup different proxy hosts and ports to GooglePlayAPI clients ? If so, how it could be achieved ?
Thanks!
Hi,
I am able to Checkin, Register with the Credentials but when i try to download an app i get error.
Error retrieving information from server. [RPC:S-5:AEC-0]·♣
Hi,
I am using the search function in the TestGooglePlay.java file. When I search for any tag, I can fetch maximum 20 results. Is that a restriction from Google Play?
SearchResponse searchResponse = service.search("calendar");
Trying to wrap my head around the whole Android_ID, autogeneration etc...
We have a server with the google play crawler where I can successfully serve all the operations via REST endpoints to be consumed by clients connecting to the REST API.
Obviously the user / password constructor then checkin() is not an option for this case.
We want to use the clients access to google accounts to provide our Android id, secretToken and cd2m auth token so each client has it's own GooglePlayAPI instance associated to the account that use for the requests.
What is the recommended flow approach to use the GooglePlayAPI class without a user and pass?
The checkin() call implies you have a password instance variable.
Any ideas on how to implement such a flow? thanks in advance and great job.
Hi,
I´ve problems with the use of "java -jar googleplay.jar list".
I don´t know how to use it right.
I want to have (for example) the top 50 apps of each category (free apps).
Here´s my command:
"java -jar gooleplay.jar -f crawler.conf list -s tools category"
=> I get: "↕6↕4Error retrieving information from server. [DF-LA-02]↨☻"
But I can download app without probelems...
Can anyone give me an example?
Thanks :-)
Could you added supporting of Google Play Movies?
It will be a great bunch of tools.
As I understand need only protobuf for that?
I have a Samsung Galaxy S3 running Android 4.2.2 (CyanogenMod 10.1). I'm using the android ID from the that device, with the Google Apps account login and password associated with it. I can search for apps, but the problem is that it is only showing apps that are available for Android 1.1, and not all the apps available for my device. Here is an example:
➜ googleplay git:(master) ✗ java -jar googleplaycrawler-0.2.jar -i myandroididhere -e [email protected] -p mypasswordhere search gmail
Title;Package;Creator;Price;Installation Size;Number Of Downloads
Backup to Gmail;net.cpedia.backup2gmail;cpedia Mobile;$1.99;1296011;50,000+
Gmail Unread Count OLD;se.alexanderblom.gmailunreadcount;Alexander Blom;Free;321781;500,000+
Cloud Print My Gmail;com.khymaera.android.cpmg;Khymaera;Free;579057;10,000+
backup SMS MMS Photo to Gmail;com.xmobileapp.xarchive;ianXian;$1.99;419239;1,000+
SMS Backup (Gmail);mobi.eapps.SMSBackupFree;WooBoo;Free;263897;10,000+
SMS Backup or Save to Gmail;com.jeff.app.smsforgmail;Think Studio;Free;326301;10,000+
Gmail Tasks;funwithapps.gmailtasks;Fun With Apps;Free;14207;10,000+
Contacts Backup(Gmail);com.startrek.ContactsBackupGmail;tarzanapp;$0.99;292540;500+
Gmail Spam Checker;net.jundu.android.spamchecker;JUNDU;Free;83999;5,000+
SMS to GMAIL;com.mpgroupslovakia.smstogmail;M&P Group Slovakia;Free;277915;1,000+
Fast Email and Gmail;com.tenromans.fastemail;Top Download;$0.99;45389;500+
GTD GMail Collect;net.ronincoders.automail;Salvamaine.com;Free;264365;1,000+
SMS Backup(Gmail);com.tarzanapp.SMSBackupGmail;tarzanapp;$0.99;271003;100+
Gmail Priority;com.netcompss.gpri.gmail3;NetComps LTD;$1.99;227903;100+
eNotify Gmail;com.netcompss.enotify3.gmail3;JY Team NetComps LTD;$1.99;350772;50+
FreeNote, note everything;com.suishouxie.freenote;flyable;Free;629838;1,000,000+
LED me know BETA;com.p0w4.ledMeKnow;p0w4p0ty;Free;379387;100,000+
Facebook;com.facebook.katana;Facebook;Free;3091271;100,000,000+
Pandora® internet radio;com.pandora.android;Pandora;Free;2124677;50,000,000+
Documents To Go 3.0 Main App;com.dataviz.docstogo;DataViz, Inc.;Free;4591508;10,000,000+
Gmail itself doesn't even show up in my search results. All of these apps are for Android 1.1. I appreciate any advice!
bad security token
Error 401
I'm new to google play crawler and java, my server is on Amazon cloud, US Region.
I update /home/ubuntu/google-play-crawler-master/googleplay/crawler.conf with my gmail account and password.
root@ubuntupc:/home/ubuntu/google-play-crawler-master/googleplay# java -jar googleplay.jar -f crawler.conf checkin
Your account succesfully checkined!
AndroidID : 33dd2d49569f3f23
SecurityToken : 4eb656111550c479
I update /home/ubuntu/google-play-crawler-master/googleplay/crawler.conf with above AndroidID and SecurityToken
root@ubuntupc:/home/ubuntu/google-play-crawler-master/googleplay# java -jar googleplay.jar -f crawler.conf register
A device is registered to your account! You can see it at "https://play.google.com/store/account" after a few downloads!
root@ubuntupc:/home/ubuntu/google-play-crawler-master/googleplay# java -jar googleplay.jar -f crawler.conf search games
Title;Package;Creator;Price;Installation Size;Number Of Downloads
Index: 0
root@ubuntupc:/home/ubuntu/google-play-crawler-master/googleplay# java -jar googleplay.jar -f crawler.conf search facebook
Title;Package;Creator;Price;Installation Size;Number Of Downloads
Index: 0
root@ubuntupc:/home/ubuntu/google-play-crawler-master/googleplay# java -jar googleplay.jar -f crawler.conf search play
Title;Package;Creator;Price;Installation Size;Number Of Downloads
Index: 0
Any suggestion on why I always get Index:0 ?
Hey, this is a great project, thanks.
I have tried to get description for applications with bulk query but the response did not contain this field.
I am just wondering how it works without android id ? I'm really surpised about that. I just provided google username an password and that's it. Is it normal to work without valid android id ? Thanks.
Thanks a lot Akdeniz for the great work!
As some other people already mentioned, checkin no longer works now. So if I want to run usegcm, how do I find out the security token? service.getSecurityToken() returns null after login().
BTW, if I want to get the APK files for paid APPs, is usegcm the only feasible option? service.download() fails for paid APPs.
Hi, Akdeniz
Thanks for your goolge play api.
I try to use it to crawl user reviews of apps.
I find that the method getAuthorName() for retrieving the author name of user reviews seems not work. I'm not sure if there are some bugs.
Best,
Ning
Is it possible to grab the developer email address from the API?
I need Google Play data(for example, reviews) in multiple localizations. I want to parse reviews in multiple threads, so in each thread I
When runned in one thread, code works perfectly, but problems appears in multithreaded mode - reviews are requested from incorrect locale. I don't want to synchronize these two actions by mutex or smth else - it destroys the idea of using threads. There was an idea about cloning instance of GooglePlayAPI
object for each thread, but it is unclonable.
How can I use library in multithreaded mode? Am I missing something? Maybe it's possible to set localization for each request?
My environment: jruby
, sidekiq
as threads manager. Yep, I use library from ruby.
I am using default code and in TestGooglePlay.java file, there is one method testBulkDetails() which downloads details for following packages.
"com.mobulasoft.criticker",
"com.cricbuzz.android",
"com.sticksports.stickcricket",
"com.indiagames.cricketfever".
But the problem is it only downloads information about first two packages, not for the last two ones. I have tried to debug the code what could have been a problem. The reason I found was that
service.details() and service.bulkdetails() methods cannot fetch version and installation size information about the packages from Google Play. Any idea, what should I try else? Thanks.
When I tried checkin or usegcm command, it showed the following message:
This is supposed to be overridden by subclasses.
My shell command is:
java -jar google-play-crawler/googleplay/googleplaycrawler-0.3.jar -f conf/crawler.conf checkin
The following request
POST /auth HTTP/1.1
Content-Length: 278
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Host: android.clients.google.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.2.2 (java 1.5)
Does not return a response anymore, Is this happening for anyone else?
Hi,
I tried to figure out how to download paid apps, but i was not able to find the way to do it.
It would be helpful if you can tell me how this flow works
Thanks
Is it possible to get developer's information, for example name and email via service.bulkDetails(packageNames).
I tried following code and it always empty (except packageName):
BulkDetailsResponse bulkDetails = service.bulkDetails(packageNames);
List bulkDetailsEntryList = bulkDetails.getEntryList();
for (BulkDetailsEntry bulkDetailsEntry : bulkDetailsEntryList) {
DocV2 docV2 = bulkDetailsEntry.getDoc();
DocumentDetails docDetails = docV2.getDetails();
AppDetails appDetails = docDetails.getAppDetails();
System.out.println(appDetails.getPackageName());
System.out.println(appDetails.getDeveloperName());
System.out.println(appDetails.getDeveloperEmail());
}
but with a single call like this:
DetailsResponse details = service.details(packageNames.get(0));
AppDetails appDetails = details.getDocV2().getDetails().getAppDetails();
System.out.println(appDetails.getDeveloperName());
System.out.println(appDetails.getDeveloperEmail());
everything works fine.
Is it possible to get developers bulk details ?
Thanks.
Tried installing the project several times.
At first, thought it was a scala/sbt version problem but after several tries, I always end up with this error
| private Queue filteredMessages = new ConcurrentLinkedDeque<>()
Any suggestions?
OS: Ubuntu 12.04
In TestGooglePlayCrawler.java, login() will be failed by using the android-id generated after run checkin().
Listcommand also failed.
Recently I could not download any apps through this library, It's always return null for getDownloadAuthCookie(0) and download get failed!
BTW DocV2 is no longer working and I have to use DocV1!
any ideas?
I used your code before to build a play crawler but it failed several days before because of the checkin problem. And then I updated with the newest one you supplied. I got this error message from TestGooglePlayCrawler. Do you know why? Thanks!!
[TestNG] Running:
/tmp/testng-eclipse--1719814961/testng-customsuite.xml
FAILED: shouldCheckin
java.lang.VerifyError: class com.akdeniz.googleplaycrawler.GooglePlay$AndroidCheckinRequest overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.akdeniz.googleplaycrawler.Utils.generateAndroidCheckinRequest(Utils.java:167)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.checkin(GooglePlayAPI.java:156)
at com.akdeniz.googleplaycrawler.TestGooglePlayCrawler.shouldCheckin(TestGooglePlayCrawler.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:335)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:330)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)
Hi Akdeniz,
You are awesome!
I am trying to get all the reviews for an app. However, it gives me an error "Too many results requested.". Could you please tell me how could I avoid this? For example this app "de.lotum.whatsinthefoto.us" has over 200K reviews but I can get only some of them :(
ReviewResponse reviews = service.reviews("de.lotum.whatsinthefoto.us", null, 500, 20);
GetReviewsResponse response = reviews.getGetResponse();
Exception in thread "main" com.akdeniz.googleplaycrawler.GooglePlayException: ����Too many results requested.*��»�
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeHttpRequest(GooglePlayAPI.java:490)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeGet(GooglePlayAPI.java:481)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeGETRequest(GooglePlayAPI.java:385)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.reviews(GooglePlayAPI.java:352)
at com.akdeniz.googleplaycrawler.test.TestGooglePlay.reviewsCommand(TestGooglePlay.java:79)
at com.akdeniz.googleplaycrawler.test.TestGooglePlay.main(TestGooglePlay.java:62)
In order to parse apps and comments in differents languages we need to provide different locales per request. How it could be implemented with this library ? Thanks.
I have today responses
java -jar googleplaycrawler-0.2.jar -f crawler.conf search games
<HTML>
<HEAD>
<TITLE>Not Found</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000">
<H1>Not Found</H1>
<H2>Error 404</H2>
</BODY>
</HTML>
Hi
First, thank you for great job.
I want to ask, how i can change country of google play from which google return category application list. Changing only localization return the same list of applications. I found in googlePlayApi.java only one country string - device_contry in checkin. Is that it? If yes, should I "rechekin" for each country which i want to crawl?
Thank for reply
First of all, thanks for your library. It is extremely useful and easy to use.
Is it possible to add a new recommendation type - similar?
Thanks.
Is there a way to list apps in collections like these?
https://play.google.com/store/apps/collection/promotion_3000147_play_picks_phone
Both the Google Play app and site show those but I don't see to be able to fetch those with either browse or list methods.
Also if there is is there a way to list all the available collections
Thanks!
i used https://code.google.com/p/android-market-api/ but it's old and sometimes buggy when trying to download some apps.
this api is great, but missing search for free apps.
[info] Compiling 20 Java sources to /Users/ahikmat/Documents/kh/crawler/google-play-crawler-master/target/scala-2.9.2/classes...
[error] /Users/ahikmat/Documents/kh/crawler/google-play-crawler-master/src/main/java/com/akdeniz/googleplaycrawler/gsf/MessageFilter.java:18: illegal start of type
[error] private Queue filteredMessages = new ConcurrentLinkedDeque<>();
[error] ^
[error] 1 error
error javac returned nonzero exit code
[error] Total time: 135 s, completed May 27, 2014 10:54:06 AM
Hi,
first of all great job!
I was wondering how can we parse the category of an app and any other available details, after the search result returns?
I tried appDetails.getAppCategory() by randomly guessing that there should be something like that but it just prints Index= Size=0.
Where can I find the API for AppDetails and DocV2?
Thanks!
This is not an issue or a bug, I just want to know that how much application can we download via this library?
BTW thanks to @Akdeniz for this awesome open source library.
Hi sir,
I have been trying to build the code following the steps specified. My development platform is osX and other sbt and maven details are following:
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 19:21:28+0530)
Maven home: /Users/tushar/Downloads/apache-maven-3.0.5
Java version: 1.6.0_65, vendor: Apple Inc.
Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "mac os x", version: "10.9", arch: "x86_64", family: "mac"
The sbt step runs fine but the maven step is giving the following problem:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project googleplaycrawler: Compilation failure -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project googleplaycrawler: Compilation failure
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:862)
at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:129)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
... 19 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
I would be glad if you could help me solve this issue.
Hi,
Thank you for your api, it worked great.
But recently, I have found an error when checkin, please check the message shown below. What is the reason? Thank you very much.
Exception in thread "main" com.akdeniz.googleplaycrawler.GooglePlayException:
<TITLE>Not Found</TITLE>at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeHttpRequest(GooglePlayAPI.java:489)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executePost(GooglePlayAPI.java:449)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executePost(GooglePlayAPI.java:429)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.loginAC2DM(GooglePlayAPI.java:163)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.checkin(GooglePlayAPI.java:148)
at com.akdeniz.googleplaycrawler.test.TestGooglePlay.main(TestGooglePlay.java:38)
Is there any way to tell if an app is not available due to country restrictions? I have an array of ~10 devices to try downloading an app. I know I could check for checkoutRequired to see if its a price restriction, but I don't know how to account for connectivity issues vs country availability.
Hi,
how can I set a proxy in the "TestGooglePlay.java" class?
I set it in the crawler.conf, but I think this class don´t gets anxy information from the conf-file, right?
Thanks
Hello,
Thank you very much for the good work first.
I am trying to get a list of apps, which is in large scale. My scritp is to get 50 apps of each category in a time to the total number 2000.
The problem is that as the offset in the search is larger than 500. The search will come across a failure. I do not think it is caused by the blocking of my IP address. In my manual test, the successful search (offset less than 500) follows tightly with the search with failure. Also, in my script, it sleeps for 10 mins between two searchs.
I appreciate your efforts on reading this problem.
Best regards,
Zhengyang
java -jar googleplaycrawler-0.3.jar -f crawler.conf list BOOKS_AND_REFERENCE -s apps_topselling_free -n 50 -o 200
[success]
java -jar googleplaycrawler-0.3.jar -f crawler.conf list BOOKS_AND_REFERENCE -s apps_topselling_free -n 50 -o 500
[failure]
Hi, Akdeniz
Really grateful for your google-play-crawler, I can checkin and download apps with a new gmail account without 'Android Id' now.
But I got a problem when access https://play.google.com from the browser, after login with the same account just checkin, it did not display my account was binded with the Galaxy S3, it say "You don't have any devices", and then I can not download any apps from the browser.
So can you help me, waitting for your reply....
I got this error:
Exception in thread "main" com.akdeniz.googleplaycrawler.GooglePlayException:
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeHttpRequest(GooglePlayAPI.java:522)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeGet(GooglePlayAPI.java:513)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeDownload(GooglePlayAPI.java:356)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.download(GooglePlayAPI.java:320)
at com.akdeniz.googleplaycrawler.TestGooglePlayCrawler.shouldDownload(TestGooglePlayCrawler.java:149)
at com.akdeniz.googleplaycrawler.TestGooglePlayCrawler.main(TestGooglePlayCrawler.java:163)
Can you help me with this ? Thank you!
Hi,
my test google account for this api was disabled by google.
I think because I used this API to often? Is that possible?
How can I avoid this in future and should I just reactivate my account by let sending me a verification code via sms?
thanks
I have been trying this out, but I'm consistently getting the same response. A bulk detail request for any number of packages (2 and above) will result in a response that has the correct number of documents returned, but only the first one is actually filled with data.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.