mattbdean / jraw Goto Github PK

I'd like to be able to disable caching. When I poll for "new" comments, I only see an updated response every 2-3 minutes while using JRAW. If I hit the same URL in my browser I see it continually updated.

Looks like OKHttp's CacheControl should allow for this, but not exactly sure where that should be used in JRAW. If you have any tips let me know, I'd be glad to create a pull request.

In upload_docs.sh, a mechanism is in place to make sure that the docs for a specific commit aren't uploaded twice to the same location. This is currently being done by cloning gh-pages and checking if a specific folder exists. Instead, an HTTP request could be sent to the github.io site at the beginning of the script to prevent the Travis build from doing unnecessary work.

Here is how the request could be sent in cURL:

curl -s -o /dev/null -w "%{http_code}" -L -I http://thatjavanerd.github.io/JRAW/docs/git/{short_hash}/index.html

(source)

This sends a HEAD request to the index.html page of the commit to test and will result with the output of "404" or "200".

If the code is a 404 Not Found, then we know not to continue. If else, it's all good and we can upload the docs.

The OAuth2 wiki page doesn't go into much detail on how to re-gain access to the previously logged-in user without having to have the user re-allow the application access to their profile.

Currently, I ask the user for a username + password, and if the UserChallengeTask completes correctly, I store the two values to re-authenticate on the next time the application is opened. I can create a new credentials object using the username + password, but how do you tell the RedditClient to log in using those details and skip the permissions screen (that they have already accepted)?

Thanks!

For some reason this code throws a NullPointerException at for(Comment comment : s.getComments()) even though the submission has comments.

SubredditPaginator subreddit = new SubredditPaginator(reddit, "all");
            subreddit.setLimit(100);
            subreddit.setSorting(Sorting.RISING);
            while(subreddit.hasNext()){
                Listing<Submission> submissions = subreddit.next();
                for(Submission s : submissions)
                {
                    System.out.println("Submission: "+s.getTitle());
                    for(Comment comment : s.getComments())
                    {
                        System.out.println("-Comment: "+comment.getAuthor());
                    }
                }
            }

As matt5188 said,

It appears to be due security restrictions that Travis imposes on secure environment variables when handling pull requests from a forked repo. Not sure the best way to handle this scenario - this is the first time I've worked with Travis. I'll read into it.

A solution to this would be to run a custom TestNG suite if [ "${TRAVIS_PULL_REQUEST}" = "x" ] where x is the PR number.

If this method is used, then the PR would still have to be tested locally before merging to make sure that all the features that depend on authentication still work.

See here for Travis' docs on security in PRs

This will fix net.dean.jraw.test.AccountManagerTest.testUpdatePreferences().

There is a shocking lack of flair support in this library. The basics (at a minimum) need to be covered.

• POST /r/{subreddit}/api/flairselector List username flairs
• POST /r/{subreddit}/api/selectflair Choose flair
• POST /r/{subreddit}/api/setflairenabled Enable/disable user flair

Any endpoints under the modflair are helpful, but not required.

This issue is similar to #7, except that addDeveloper() is returning an ApiException whose code is DEVELOPER_ALREADY_ADDED, even though removeDeveloper() was executed right before that call.

Here is the relevant code.

Hi there,

I'm trying to use JRAW on android, but it crashes on any device older than android 4.4 with a NoClassDefFoundError, as it can't find the StandardCharsets Class (Which was added in API 19 https://developer.android.com/reference/java/nio/charset/StandardCharsets.html).

Is there any way to fix this?

Best Regards,
Michael.

See here for how this can happen.

Now that Reddit supports HTTPS sitewide, a new field is returned in the /api/login response: "need_https". If this value is true, any test that is executed while a user is logged in will fail because it is executed over plain HTTP.

Here is a sample response:

{
    "json": {
        "errors": [],
        "data": {
            "need_https": false,
            "modhash": "...",
            "cookie": "..."
        }
    }
}

Some comment trees have a "continue this thread" option. Currently JRAW doesn't expand the full comment tree:

see e.g.

http://www.reddit.com/r/test/comments/2onit4/test_depth_0/

(scroll down to depth 10)

This is a known issue and has recently been fixed on PRAW, see e.g.

http://www.reddit.com/r/redditdev/comments/2onkd8/praw_maximum_depth_of_subtrees/

Right now, it would be easy to ignore the user's wishes about requiring HTTPS by calling RestClient.setHttpsEnabled(boolean). Disabling HTTPS on an account like this will cause errors to be sent back to the client.

This is the code i've tried:

HttpLogger logger = new HttpLogger(JrawUtils.logger());
logger.disable(HttpLogger.Component.REQUEST);
logger.disable(HttpLogger.Component.RESPONSE);

I've added the above snippet at various points in the code with no luck (INFO JRAW messages are still displayed)

I'm using the latest release build v0.6.1

Several issues (#29, #44, and #66) have already been opened about Submission.getComments() returning null if it was retrieved by a SubredditPaginator. Some documentation should be added to the SubredditPaginator class explaining why this happens and what the user should expect from this method.

See here for an explanation.

The current method for parsing JSON into Objects is really clunky (see here). This could be drastically improved by using Jackson databind and annotations libraries.

Even when I know that the submission has comments and getCommentCount() tells me the right amount of comments, getComments() always returns null.

For example, under the "Implemented?" header, ENDPOINTS.md lists RedditClient.login(String, String) as simply RedditClient.login(). This could give anyone who isn't familiar with the library the idea that providing a username and password is done somewhere else in the code.

Due to the switch from referencing the source to the Javadoc, @EndpiontImplementation annotations should be limited to methods that have a privacy of at least protected, since private methods aren't shown in Javadoc.

According to this StackOverflow question, char arrays are preferred over Strings for password handling because it reduces the window that an attacker could obtain the password before GC kicks in.

To implement this, one might separate TestUtils.getCredentials() into String getUsername() and char[] getPassword() methods and then make a login(String, char[]) method in RedditClient.

The fact that many methods in AccountManager return a RedditResponse means that whomever is using the library might feel the need to do something with this response, when that is not the case at all. Whatever data that is shown in these responses should be gleaned and the methods should return void.

I use the following code in Android:

//In Activity:
        ConnectionTask task = new ConnectionTask();
        Credentials credentials = Credentials.standard("myusername", "mypassword");
        task.execute(credentials);


//Custom Async:
public class ConnectionTask extends AsyncTask<Credentials, Void, LoggedInAccount> {
    private final String TAG = getClass().getName();

    @Override
    protected LoggedInAccount doInBackground(final Credentials... params) {
        RedditClient reddit = new RedditClient("MY-JRAW-REDDIT-ANDROID-APP");

        try {
            LoggedInAccount loggedInAccount = reddit.login(params[0]);

            Log.e(TAG, "getfulname: " + loggedInAccount.getFullName());
            Log.e(TAG, "com karma: " + loggedInAccount.getCommentKarma());
            Log.e(TAG, "link karma: " + loggedInAccount.getLinkKarma());

            return loggedInAccount;
        } catch (Exception ne) {
            ne.printStackTrace();
            return null;
        }
    }

    @Override
    protected void onPostExecute(final LoggedInAccount loggedInAccount) {
        if (loggedInAccount != null) {
            Log.e(TAG, "getfulname: " + loggedInAccount.getFullName());
            Log.e(TAG, "com karma: " + loggedInAccount.getCommentKarma());
            Log.e(TAG, "link karma: " + loggedInAccount.getLinkKarma());
        }
    }
}

and get the following Exception:

01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at net.dean.jraw.models.JsonModel.data(JsonModel.java:65)
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at net.dean.jraw.models.JsonModel.data(JsonModel.java:43)
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at net.dean.jraw.models.Thing.getFullName(Thing.java:37)
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at net.dean.jraw.RedditClient.login(RedditClient.java:160)
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at com.natieklopper.appreddit.util.ConnectionTask.doInBackground(ConnectionTask.java:21)
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at com.natieklopper.appreddit.util.ConnectionTask.doInBackground(ConnectionTask.java:13)

Debug Object (me) in RedditClient at line 160:

LoggedInAccount {getCommentKarma()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], getCreated()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'org.codehaus.jackson.JsonNode org.codehaus.jackson.JsonNode.get(java.lang.String)' on a null object reference], getCreatedUtc()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'org.codehaus.jackson.JsonNode org.codehaus.jackson.JsonNode.get(java.lang.String)' on a null object reference], getFullName()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], getId()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], getLinkKarma()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], getModHash()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], getType()="ACCOUNT", hasGold()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], hasMail()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], hasModMail()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], hasVerifiedEmail()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], isFriend()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], isMod()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], isOver18()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference]}

Anything I'm missing here?

A task in build.gradle could easily help new devs get started with JRAW. It should:

1. Take a username, password, password verification, optional email, and a captcha attempt and register a new user.
2. Create a OAuth2 app with type 'script' (AccountManager.createOrUpdateApp())
3. Write the username, password, client id, and client secret to credentials.json using Jackson
4. Create a subreddit
5. Create a multireddit whose name is not jraw_testing
6. Submit a selfpost to /r/jraw_testing2

JRAW could be included into the buildscript's dependencies to make this easier.

When I run this code:
submissionSearchPaginator.setTimePeriod(TimePeriod.valueOf("DAY")); //I do valueOf because the value might be different//
I still get results from hundreds of days before. The method seems not to work.

According to this thread, /u/kemitche has now deprecated authentication via POST /api/login.

Use of the API when authenticated via cookies is deprecated and slated for removal. All API clients MUST convert to authenticating to the reddit API via OAuth 2 by August 3, 2015. After that date, reddit.com will begin heavily throttling and/or blocking API access that is not authenticated with an OAuth 2 access token. Yes, this applies to "logged out" access to the API. For API access without a reddit user, please use Application Only Authentication to get an access token.

I can foresee a few changes to the codebase in response:

1. RedditClient and RedditOAuth2Client will become merged.
2. Credentials will have a new factory method for Application Only authentication (using the API without a user)
3. RedditClient.login(Credentials) will become mandatory, since some form of OAuth2-based authentication must happen (whether that's an app or application-only) before the API can be reached.

The other topic mentioned in this post is the naming restrictions on API clients.

1. We're asking API clients to not use the word "reddit" in their name except in the phrase "for reddit", e.g., "My cool app for reddit"
2. We're asking "commercial" API consumers to register with us.

Since this is an open source wrapper and the licensing page defines "commercial" as "if you are earning money from it, including via in-app advertising or in-app purchases. Open source use is generally considered non-commercial," so the project should still be able to maintain the same name.

Currently, all links to endpoints use the format https://www.reddit.com/dev/api#{endpoint}. Since ENDPOINTS.md now categorizes endpoints by their OAuth2 scope, the path should be /dev/api/oauth instead of /dev/api

In MultiRedditManagerTest's testCreate() method, a multireddit of a given name is deleted by calling manager.delete(name) if a multireddit of the same name already exists. This call throws no ApiException, so logically we can assume that the multireddit has been deleted.

However, after the code flow gets to manager.create(), the call throws an ApiException whose code is MULTI_EXISTS. This is strange because we have already insured that that specific multireddit did not exist before executing the request.

See here for the relevant code.

A constant of 5 seconds is fine for testing purposes, but since this is a library revolving around networking, this value should be configurable.

Currently there is no way to use the customizability provided by the HttpLogger class because RestClient does not expose it. A simple getter method should do.

According to this, ssl.reddit.com should be used for "logging in, OAuth authorizations, and preferences". Using https://www.reddit.com is preferred to https://ssl.reddit.com for "normal" requests.

Having support for Java 7 would make the library usable for Android apps and other projects that aren't ready to take the switch to Java 8.

Pros:

• Support for Android
• According to Plumbr, Java 7 is the most popular version as of May 8, 2014
• Test both Oracle JDK 7 and OpenJDK 7 on Travis-CI as opposed to just Oracle JDK 8
• (IMHO) better looking Javadoc

Cons:

• No default methods for interfaces (Created, Distinguishable, etc.)
• No lambdas/streams (although the standard for loop seems superior)
• No java.time classes (e.g. LocalDateTime in RestClient)

Currently usernames and passwords alike are stored in plaintext strings. According to this StackOverflow answer, credentials should be encrypted, "only decrypting them temporarily during the authentication process." Using something as simple as a debugger on an instance of Credentials could give an attacker an all-access pass to whatever account is being used.

In JsonModel's toString() method, if a return value of a @JsonInteraction method is a subclass of JsonModel, the representation of that model is printed as "[ClassName]". For example:

CustomModel {getFoo()="Object@12345", getListing()=[Listing]}

This isn't very descriptive. It would be a lot more helpful if Listings were viewed as Listing[size] or some other variant.

A sample output could look like this:

CustomModel {getFoo()="Object@12345", getListing()={Listing getSize()="25"}}

RenderStringPair's only purpose is to contain two versions of the same data - one raw Markdown text, and the other the HTML version. The HTML version is useless to the API, so that can be safely discarded.

I am getting the front page perfectly with a SubredditPaginator, but for every submission in the Listing, getComments() returns null.

I can't seem to find any method to get subscribed subreddits (preferably returning a List of sub names). Does one exist?

Thanks!

I'm getting the following error
net.dean.jraw.http.NetworkException: Request returned non-successful status code: 400 Bad Request

when running the following code:

OAuthHelper helper = reddit.getOAuthHelper();
helper.setRefreshToken("Token received from UserChallenge");

OAuthData finalData = helper.refreshToken(Credentials.installedApp("my key", "http://ccrama.me"));
reddit.authenticate(finalData);

Is this an issue with JRAW or an issue with how I'm trying to refresh my token?

Thanks!

By using a framework such as Mockito to mock certain classes, we can cut down on the amount of requests we have to send over the network and reduce testing time. Granted, we will still have to send some requests to properly test them. Fixtures can be created in the src/test/resources/ directory and will be returned when mocked classes request a certain resource. See here for an example.

This will require a huge undertaking and might best be accomplished in smaller chunks.

The only time it is necessary for the client to ask for a username/password is when the client is using application-only OAuth or the app type is 'script'. Otherwise, the browser should handle authentication.

The new methods signatures should be

public static Credentials installedApp(String clientId, String redirectUrl)

and

public static Credentials webapp(String clientId, String clientSecret, String redirectUrl)

I'm having an issue where I get a Network exception when running

SubredditPaginator sub= new SubredditPaginator(reddit, "subreddit");
return sub.next();


Stacktrace:

Caused by: net.dean.jraw.http.NetworkException: Request returned non-successful status code: 403 Forbidden
at net.dean.jraw.http.RestClient.execute(RestClient.java:134)
at net.dean.jraw.RedditClient.execute(RedditClient.java:152)
at net.dean.jraw.paginators.Paginator.next(Paginator.java:107)
at net.dean.jraw.paginators.SubredditPaginator.next(SubredditPaginator.java:44)
at net.dean.jraw.paginators.Paginator.accumulate(Paginator.java:127)
at net.dean.jraw.paginators.Paginator.accumulateMerged(Paginator.java:135)


From some googling, it seems as if OAuth was messed up somewhere, even though I have not implemented logging in/OAuth (yet). What could be the issue?

Thanks!

EDIT: Just FYI, I ran the code with "all" instead of "subreddit"

Reddit lets you choose a week for a time period, but JRAW only has HOUR, DAY, MONTH, YEAR, and ALL

I'm using JRAW in an Android Wrapper, it is a lib to my Android app I'm working on.
I would like to add tests to my JRAW Android Wrapper Library using Wiremock, for this I would need to be able to set the host via RedditClient.setHost() and setSpecialHost(). Also to set the Port and HTTP/HTTPS.

Can these please be added?
I don't wan't to have the source of JRAW and edit that, at the moment I'm just using it via Gradle as a library.

Thank you.

Some new preferences have been added since the class was last updated, such as forcing HTTPS and monitoring username mentions.

https://ssl.reddit.com/api/v1/authorize?state=hg1nscn1581r1hseddf1ig5ovr&duration=permanent&scope=identity%2C&response_type=code&client_id=N2Lpad7lBpi4kA&redirect_uri=oauth%3A%2F%2Fasne

caused by redundant %2C (encoded ',') at end of scope=identity parameter.

Hello,

I encountered a bug where without read timeout set my application hangs on socketRead0. I could not set this timeout without extending your class, because OkHttpClient is not accessible from outside.

In AccountManager, some methods have names that are just the action (vote(), hide(), delete()), and others mimic the style of a setter method (setSubscribed(), setSticky(), setSaved()). There should only be one style of method in this class.

Currently only 'script' apps are allowed, although Credentials provides a way to create instances for web and installed apps.

This should be relatively painless to implement in OAuth2RedditClient's login(Credentials) method.

See here for an overview of the requests needed to accompilsh this.

The method "loadMoreComments(...) doesn't work correctly. The new comments that are loaded aren't integrated into the tree.

(apologies if this is not the right place for this kind of feedback )

Returning java.net.URL does two things:

• Opens up the possibility of MalformedURLExceptions when the post's ID has characters that are not allowed in a URL. This could happen when when submission.getShortUrl() is called directly, or when submission.toString() is called.
• Deviates from the standard of returning strings for any URIs or URLs returned by the API

Hosting all of the docs for every single commit creates an unnecessarily large Git repo. Most people won't even look at these. At the very minimum, we should only host Javadoc for the latest passing commit.

The new URL to access these docs could be /git/ instead of /git/latest.