jcabi / jcabi-http Goto Github PK

Puzzle 19-aa7a7497 in src/main/java/com/jcabi/http/mock/MkContainer.java:64-66 has to be resolved: Let's introduce a utility class MkQueryMatchers which contains convenience methods for matching MkQuery objects. This is primarily intended for use with the conditional MkContainer.next() methods. For example:

Puzzle 19-a877bdce in src/main/java/com/jcabi/http/mock/MkContainer.java:87-89 has to be resolved: Let's add methods take(Matcher ) and takeAll(Matcher ), and a utility class MkAnswerMatchers. Intended usage is as follows:

Let's get rid of commons-lang3. It's a small library, would be better to have it deps-free

Thanks for this very useful library - I was getting version conflicts with anything that uses ApacheHttp, and this gave me a very quick solution.

There does seem to be a dependency missing from your "optional dependencies" page, for the the Lombok annotations.

In Gradle speak:

compile 'org.projectlombok:lombok:1.14.8'

I'm currently using:

compile 'com.jcabi:jcabi-http:1.10.1'
compile 'javax.json:javax.json-api:1.0'
compile 'org.glassfish:javax.json:1.0.4'

JsonResponse is now relying on a proper Unicode formatting of JSON documents (see line 95). When JSON contains non-Unicode characters, the class throws a runtime exception. For example, try this document (which is a valid JSON according to RFC 4627):

{"test": "\u0000"}

At the moment, we have a method RestResponse.assertBody(Matcher<String>), which checks if the response body String corresponds to the given matcher.

Let's add an equivalent assertBinary() method for checking binary content. After all, we have methods to obtain the response in either format (Response.body() and Response.binary() respectively); it would be great if we were able to have convenience methods for verifying the body content in binary as well.

JsonResponse.json() should be more verbose when parsing breaks. It has to log the content of JSON response to slf4j

Let's introduce a new utility class MkQueryMatchers, in order to enable conditional answers in MkContainer, for example:

MkContainer container = new MkGrizzlyContainer();
  .next(
    new MkAnswer.Simple("hello, world!"),
    Matchers.allOf(
      MkQueryMatchers.hasHeader(
        "Content-Type",
        Matchers.equalTo("text/plain")
      ),
      MkQueryMatchers.hasBody(
        Matchers.containsString("say hello")
      )
    )
  )
  .start();

This means that the answer will be returned only if the request contains header Content-Type, which is equals to "text/plain" and request body contains "say hello".

By default, when method next(MkAnswer) is used (with one argument only), an everything-matching instance of Matcher<MkQuery> is used.

Besides that, let's add methods take(Matcher<MkAnswer>) and takeAll(Matcher<MkAnswer>), and an utility class MkAnswerMatchers:

MatcherAssert.assertThat(
  container.takeAll(
    MkAnswerMatchers.hasHeader(
      "Content-Type",
      Matchers.equalTo("application/json")
    )
  ),
  Matchers.allOf(
    Matchers.hasSize(5),
    Matchers.hasItem(
      Matchers.allOf(
        MkQueryMatchers.hasHeader("User-Agent"),
        MkQueryMatchers.hasBody(
          Matchers.containsString("say hello")
        )
      )
    )
  )
);

This means that we're taking all requests received by the container, which were answered with responses that had a header "content-type" with a value "application/json". We're asserting that there were 5 those answers. And all of them had body "say hello" and a header "User-Agent".

When the RequestBody of a FakeRequest is set, it incorrectly uses it in the body of the ResponseBody instead of using the response body content that was specified.

For example, the following test will fail:

@Test
public void fakeRequestReturnsRequestBody() throws Exception {
    MatcherAssert.assertThat(
        new FakeRequest()
            .withBody("foo")  // Response body - should be returned
            .body().set("bar").back() // Request body
            .fetch().body(),
        Matchers.is("foo")
    );
}

Error:

java.lang.AssertionError: 
Expected: is "foo"
     but: was "bar"

Note that the following test - if the RequestBody is not set - works fine:

@Test
public void fakeRequestReturnsRequestBody() throws Exception {
    MatcherAssert.assertThat(
        new FakeRequest()
            .withBody("foo")
            .fetch().body(),
        Matchers.is("foo")
    );
}

My environment : java 8, aspectj 1.8, intellij

P.S. didnt figure out how to paste here code so adding screenshots

For example, in http://http.jcabi.com/example-request.html, the first link to Request refers to http://http.jcabi.com/apidocs-1.5.2/com/rexsl/test/Request.html. This will result in a HTTP 404 error, since we've migrated the classes from com.rexsl.test to com.jcabi.http. The link http://http.jcabi.com/apidocs-1.5.2/com/jcabi/http/Request.html works properly as expected.

Let's update the links on this page, and all our other pages, to suit the new package structure.

Let's use jcabi-matchers 1.0 instead of rexsl-test 0.13

Let's make it fully compatible with Android

Version 1.0-SNAPSHOT is mentioned in docs 2.0-SNAPSHOT.

mvn clean install -Pqulice fails with following message:

[INFO] No Checkstyle violations found in 84 files
[INFO] No PMD violations found in 71 files
[INFO] \src\changes\changes.xml: to be validated
[INFO] jcabi-aspects 0.17.1/8b0e260 started new daemon thread jcabi-loggable for
 watching of @Loggable annotated methods
[ERROR] JSR-303 validator failed to initialize: Unable to create a Configuration
, because no Bean Validation provider could be found. Add a provider like Hibern
ate Validator (RI) to your classpath. (see http://www.jcabi.com/jcabi-aspects/js
r-303.html)
[INFO] \src\site\site.xml: to be validated
[INFO] CodeNarc not required since no groovy files in d:\dev\teamed.io\jcabi-htt
p\src
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:40 min
[INFO] Finished at: 2015-01-25T22:54:57+04:00
[INFO] Final Memory: 37M/154M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.qulice:qulice-maven-plugin:0.10:check (jcabi-
check) on project jcabi-http: Execution jcabi-check of goal com.qulice:qulice-ma
ven-plugin:0.10:check failed: Non-zero exit code 1: -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please rea
d the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutio
nException

Let's add a new class AutoRedirectingWire. It should retry a request when HTTP response code is between 300 and 399. It has to have two constructors: no-args and one-arg. The argument is the maximum number of retries. By default (with no-arg ctor) the maximum is 5.

let's use it

During test compilation, maven build fails with errors like

[ERROR] /Users/roman/Desktop/project/teamio/jcabi-http/src/test/java/com/jcabi/http/wire/CookieOptimizingWireTest.java:[84,25] <U>everyItem(org.hamcrest.Matcher<U>) in org.hamcrest.Matchers cannot be applied to <java.lang.String>(org.hamcrest.Matcher<java.lang.Object>)

The documentation claims that the only dependency required is jcabi-http itself, but that seems to be true only for a very limited used. In my project I ended up needing the following in my POM:

    <dependency>
        <groupId>com.jcabi</groupId>
        <artifactId>jcabi-http</artifactId>
        <version>1.5.2</version>
    </dependency>
    <dependency>
        <groupId>org.glassfish</groupId>
        <artifactId>javax.json</artifactId>
        <version>1.0.4</version>
    </dependency>
    <dependency>
        <groupId>org.hamcrest</groupId>
        <artifactId>hamcrest-core</artifactId>
        <version>1.3</version>
    </dependency>

I think these requirements should be at least documented -- I had to find them in your POM to identify the group and artifact IDs I wanted to add. And from what I can tell there are other cases where more dependencies are required.

I am using the below dependency and putting it in pom of a simple maven web project created using archetype-generate

<dependency>
      <groupId>com.jcabi</groupId>
      <artifactId>jcabi-http</artifactId>
      <version>2.0-SNAPSHOT</version>
    </dependency>

<repositories>
      <repository>
        <id>oss.sonatype.org</id>
        <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
      </repository>
    </repositories> 

Then I have a class with the below code

public static void main(String[] args) throws IOException {
        JdkRequest jdkRequest = new JdkRequest("https://www.google.com");
        System.out.println(jdkRequest);
        String html;

        html = new JdkRequest("https://www.google.com")
          .uri().path("/users").queryParam("id", 333).back()
          .method(Request.GET)
          .header("Accept", "text/html")
          .fetch()
          .as(RestResponse.class)
          .assertStatus(HttpURLConnection.HTTP_OK)
          .body();

        System.out.println(html);
    }

It blows up on the very first line with error

Exception in thread "main" java.lang.NoClassDefFoundError: javax/ws/rs/core/UriBuilder
    at com.jcabi.http.request.BaseRequest.<init>(BaseRequest.java:148)
    at com.jcabi.http.request.BaseRequest.<init>(BaseRequest.java:126)
    at com.jcabi.http.request.JdkRequest.<init>(JdkRequest.java:231)
    at com.shashank.testJcabi.TestJcabiRequest.main(TestJcabiRequest.java:14)
Caused by: java.lang.ClassNotFoundException: javax.ws.rs.core.UriBuilder
    at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 4 more

Please suggest.

Thanks in advance !
Shashank

The page http://http.jcabi.com/optional-dependencies.html provides a full list of dependencies one can use, but this can exceed the required set for a particular use case. For example: org.jsoup:jsoup is required only if the JSoup parser is used. Adding the dependency is not needed for other use cases and unnecessarily bloats the resulting artifacts.

The ideal solution for me would be to actually have one dependency per feature, i.e. have something like com.jcabi:jcabi-http-jsoup to depend on if I want the JSoup feature. That would also avoid having to update version numbers outside of POMs (as on the page linked above).

Puzzle 1-40eeff88 in src/test/java/com/jcabi/http/mock/MkContainerTest.java:73-74 has to be resolved: Grizzly container doesn't understand same-name headers, or we don't fetch them correctly from GrizzlyRequest

Currently, CachingWire.get() has public visibility. I don't see why it shouldn't be private instead, since the Wire API is actually obtaining the Response through Wire.send(), and CachingWire.send() delegates to get() for its caching behavior.

Let's create JsoupResponse, which will normalize the HTML page after retrieval. It shouldn't expose any specific methods, just normalize the existing body()

Originally proposed by Lukas Eder at http://www.yegor256.com/2014/04/11/jcabi-http-intro.html#disqus_thread

Let's get rid of commons-io. It's a small library, and would be much better to work without big dependencies

Currently Response.as(Class<T> type) accepts any type without bounds on T.
It should be restricted to implementations of Response interface.
Something like: <T extends Response> T as(Class<T> type)
This will also help as a good self documentation of the method.

Working on #32, I build the site using "site" maven goal to check my corrections in *.md.vm files. This way I get HTMLs for the site in target/site folder. The HTMLs contain links to javadocs, like:

./apidocs-${project.version}/com/jcabi/http/response/XmlResponse.html

These links are not valid, because there are no javadocs in ./apidocs-${project.version} folder.

How should I generate the javadocs? I tried all other maven goals, none of them made javadocs.

Maybe we need such maven goal?

The current API does not seem to have any efficient way of uploading the contents of a file or other local resource.

There is RequestBody#set(byte[]), but that requires loading the full file contents into memory. The array is then cloned in the BaseBody(BaseRequest, byte[]) constructor, doubling the memory need.

While this is currently feasible for what we do, we would prefer having something like RequestBody#set(InputStream), allowing us to stream the contents of a resource directly to a HTTP server.

    String html = new JdkRequest("https://www.google.com")
      .uri().path("/users").queryParam("id", 333).back()
      .method(Request.GET)
      .header(HttpHeaders.ACCEPT, MediaType.TEXT_HTML)
      .fetch()
      .as(HttpResponse.class)
      .assertStatus(HttpURLConnection.HTTP_OK)
      .body();
  }

But there is no HttpResponse class anymore.
There should be RestResponse

Let's create TrustedWire which will ignore SSL keys

I'm getting this exception sometimes:

Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:579)
    at java.net.Socket.connect(Socket.java:528)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
    at sun.net.www.http.HttpClient.New(HttpClient.java:308)
    at sun.net.www.http.HttpClient.New(HttpClient.java:326)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1300)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
    at com.jcabi.http.request.JdkRequest$1.send(JdkRequest.java:117)
    at com.jcabi.http.wire.BasicAuthWire.send(BasicAuthWire.java:138)
    at com.jcabi.http.request.BaseRequest.fetch_aroundBody10(BaseRequest.java:197)
    at com.jcabi.http.request.BaseRequest$AjcClosure11.run(BaseRequest.java:1)

"Connection refused" is not descriptive at all to me. I'd like to see where the connection was trying to be established to, with what HTTP method, etc. Would be nice to wrap all runtime exceptions flying out of JdkRequest$1.send() in IOException and add a message there, like:

failed to POST to http://example.com

1. Checkout master branch
2. run mvn qulice:check -Pqulice
3. There are multiple Line does not match expected header line of ' * BSD License'. (HeaderCheck) messages.

This is a known bug of qulice configuration in parent-0.31 (jcabi/jcabi-parent#19), accessed via jcabi-0.16. Suggested action is waiting until jcabi/jcabi#297 fix is released and upgrading jcabi version.

Copyright sections still feature 2014 as a year.

Let's implement a simple flushing mechanism in CachingWire

I'm trying to set jcabi-http with java 1.7 and not aspectj environment, but seems there are some other dependencies that should be there in order for it to be able to operate.

I've uploaded sample project for you
https://www.dropbox.com/s/4v6wi1w361xwbio/experimental.rar

Let's add method port() to RequestURI interface

First of all, great job! This is something i was searching for long time.
Just one question. I have a problem with missing dependency. At the moment when I execute queryParam i have:

java.lang.NoClassDefFoundError: javax/ws/rs/core/UriBuilder

Do i need anything else beside compile('com.jcabi:jcabi-http:1.9.1') ?

My code:

String body = new JdkRequest(tagsPopularityUrl)
.uri().queryParam('timestamp', (new Date() - dateOffset).time).back()
.method(Request.GET)
.fetch().body()

Puzzle 1-5ecf8bb2 in src/test/java/com/jcabi/http/response/JsoupResponseTest.java:50-53 has to be resolved: Saxon complains about HTML 1.0, and I'm not sure how this is possible to fix. All we need to check in this test is that the output is a clean HTML document. Aparently JSOUP produces a broken HTML? Let's investigate and fix.

Link http://www.rexsl.com/rexsl-test/example-mock-servlet.html
located here https://github.com/jcabi/jcabi-http/blob/master/src/main/java/com/jcabi/http/mock/MkGrizzlyContainer.java#L50
is not exist anymore.

after #1 we have a broken logic. normally formatted JSON can't be parsed, for example:

{
  "test": "hello"
}

Leads to runtime exception:

javax.json.stream.JsonParsingException: Unexpected char 92 at (line no=1, column no=21, offset=20)

Let's get rid of commons-codec and use native JDK codecs

Check travis-ci logs. Let's fix.