Giter VIP home page Giter VIP logo

orange-opensource / hurl Goto Github PK

View Code? Open in Web Editor NEW
10.9K 46.0 395.0 166.77 MB

Hurl, run and test HTTP requests with plain text.

Home Page: https://hurl.dev

License: Apache License 2.0

Rust 80.65% Shell 4.08% HTML 3.02% CSS 0.31% Python 7.08% NSIS 0.23% C 0.01% Dockerfile 0.10% PowerShell 3.17% JavaScript 0.32% Roff 0.80% Vim Script 0.11% Emacs Lisp 0.13%
curl http testing-tools cli integration-testing http-client api-testing testing

hurl's Introduction

Hurl Logo

deploy status coverage Crates.io documentation

What's Hurl?

Hurl is a command line tool that runs HTTP requests defined in a simple plain text format.

It can chain requests, capture values and evaluate queries on headers and body response. Hurl is very versatile: it can be used for both fetching data and testing HTTP sessions.

Hurl makes it easy to work with HTML content, REST / SOAP / GraphQL APIs, or any other XML / JSON based APIs.

# Get home:
GET https://example.org
HTTP 200
[Captures]
csrf_token: xpath "string(//meta[@name='_csrf_token']/@content)"


# Do login!
POST https://example.org/login?user=toto&password=1234
X-CSRF-TOKEN: {{csrf_token}}
HTTP 302

Chaining multiple requests is easy:

GET https://example.org/api/health
GET https://example.org/api/step1
GET https://example.org/api/step2
GET https://example.org/api/step3

Also an HTTP Test Tool

Hurl can run HTTP requests but can also be used to test HTTP responses. Different types of queries and predicates are supported, from XPath and JSONPath on body response, to assert on status code and response headers.

Hurl Demo

It is well adapted for REST / JSON APIs

POST https://example.org/api/tests
{
    "id": "4568",
    "evaluate": true
}
HTTP 200
[Asserts]
header "X-Frame-Options" == "SAMEORIGIN"
jsonpath "$.status" == "RUNNING"    # Check the status code
jsonpath "$.tests" count == 25      # Check the number of items
jsonpath "$.id" matches /\d{4}/     # Check the format of the id

HTML content

GET https://example.org
HTTP 200
[Asserts]
xpath "normalize-space(//head/title)" == "Hello world!"

GraphQL

POST https://example.org/graphql
```graphql
{
  human(id: "1000") {
    name
    height(unit: FOOT)
  }
}
```
HTTP 200

and even SOAP APIs

POST https://example.org/InStock
Content-Type: application/soap+xml; charset=utf-8
SOAPAction: "http://www.w3.org/2003/05/soap-envelope"
<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:m="https://example.org">
  <soap:Header></soap:Header>
  <soap:Body>
    <m:GetStockPrice>
      <m:StockName>GOOG</m:StockName>
    </m:GetStockPrice>
  </soap:Body>
</soap:Envelope>
HTTP 200

Hurl can also be used to test the performance of HTTP endpoints

GET https://example.org/api/v1/pets
HTTP 200
[Asserts]
duration < 1000  # Duration in ms

And check response bytes

GET https://example.org/data.tar.gz
HTTP 200
[Asserts]
sha256 == hex,039058c6f2c0cb492c533b0a4d14ef77cc0f78abccced5287d84a1a2011cfb81;

Finally, Hurl is easy to integrate in CI/CD, with text, JUnit, TAP and HTML reports

HTML report

Why Hurl?

  • Text Format: for both devops and developers
  • Fast CLI: a command line for local dev and continuous integration
  • Single Binary: easy to install, with no runtime required

Powered by curl

Hurl is a lightweight binary written in Rust. Under the hood, Hurl HTTP engine is powered by libcurl, one of the most powerful and reliable file transfer libraries. With its text file format, Hurl adds syntactic sugar to run and test HTTP requests, but it's still the curl that we love: fast, efficient and HTTP/3 ready.

Feedbacks

To support its development, star Hurl on GitHub!

Feedback, suggestion, bugs or improvements are welcome.

POST https://hurl.dev/api/feedback
{
  "name": "John Doe",
  "feedback": "Hurl is awesome!"
}
HTTP 200

Resources

License

Blog

Tutorial

Documentation

GitHub

Table of Contents

Samples

To run a sample, edit a file with the sample content, and run Hurl:

$ vi sample.hurl

GET https://example.org

$ hurl sample.hurl

By default, Hurl behaves like curl and outputs the last HTTP response's entry. To have a test oriented output, you can use --test option:

$ hurl --test sample.hurl

A particular response can be saved with [Options] section:

GET https://example.ord/cats/123
[Options]
output: cat123.txt    # use - to output to stdout
HTTP 200

GET https://example.ord/dogs/567
HTTP 200

You can check Hurl tests suite for more samples.

Getting Data

A simple GET:

GET https://example.org

Requests can be chained:

GET https://example.org/a
GET https://example.org/b
HEAD https://example.org/c
GET https://example.org/c

Doc

HTTP Headers

A simple GET with headers:

GET https://example.org/news
User-Agent: Mozilla/5.0 
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Connection: keep-alive

Doc

Query Params

GET https://example.org/news
[QueryStringParams]
order: newest
search: something to search
count: 100

Or:

GET https://example.org/news?order=newest&search=something%20to%20search&count=100

With [QueryStringParams] section, params don't need to be URL escaped.

Doc

Basic Authentication

GET https://example.org/protected
[BasicAuth]
bob: secret

Doc

This is equivalent to construct the request with a Authorization header:

# Authorization header value can be computed with `echo -n 'bob:secret' | base64`
GET https://example.org/protected
Authorization: Basic Ym9iOnNlY3JldA== 

Basic authentication section allows per request authentication. If you want to add basic authentication to all the requests of a Hurl file you could use -u/--user option:

$ hurl --user bob:secret login.hurl

--user option can also be set per request:

GET https://example.org/login
[Options]
user: bob:secret
HTTP 200

GET https://example.org/login
[Options]
user: alice:secret
HTTP 200

Passing Data between Requests

Captures can be used to pass data from one request to another:

POST https://sample.org/orders
HTTP 201
[Captures]
order_id: jsonpath "$.order.id"

GET https://sample.org/orders/{{order_id}}
HTTP 200

Doc

Sending Data

Sending HTML Form Data

POST https://example.org/contact
[FormParams]
default: false
token: {{token}}
email: [email protected]
number: 33611223344

Doc

Sending Multipart Form Data

POST https://example.org/upload
[MultipartFormData]
field1: value1
field2: file,example.txt;
# One can specify the file content type:
field3: file,example.zip; application/zip

Doc

Multipart forms can also be sent with a multiline string body:

POST https://example.org/upload
Content-Type: multipart/form-data; boundary="boundary"
```
--boundary
Content-Disposition: form-data; name="key1"

value1
--boundary
Content-Disposition: form-data; name="upload1"; filename="data.txt"
Content-Type: text/plain

Hello World!
--boundary
Content-Disposition: form-data; name="upload2"; filename="data.html"
Content-Type: text/html

<div>Hello <b>World</b>!</div>
--boundary--
```

In that case, files have to be inlined in the Hurl file.

Doc

Posting a JSON Body

With an inline JSON:

POST https://example.org/api/tests
{
    "id": "456",
    "evaluate": true
}

Doc

With a local file:

POST https://example.org/api/tests
Content-Type: application/json
file,data.json;

Doc

Templating a JSON Body

PUT https://example.org/api/hits
Content-Type: application/json
{
    "key0": "{{a_string}}",
    "key1": {{a_bool}},
    "key2": {{a_null}},
    "key3": {{a_number}}
}

Variables can be initialized via command line:

$ hurl --variable a_string=apple \
       --variable a_bool=true \
       --variable a_null=null \
       --variable a_number=42 \
       test.hurl

Resulting in a PUT request with the following JSON body:

{
    "key0": "apple",
    "key1": true,
    "key2": null,
    "key3": 42
}

Doc

Templating a XML Body

Using templates with XML body is not currently supported in Hurl. You can use templates in XML multiline string body with variables to send a variable XML body:

POST https://example.org/echo/post/xml
```xml
<?xml version="1.0" encoding="utf-8"?>
<Request>
    <Login>{{login}}</Login>
    <Password>{{password}}</Password>
</Request>
```

Doc

Using GraphQL Query

A simple GraphQL query:

POST https://example.org/starwars/graphql
```graphql
{
  human(id: "1000") {
    name
    height(unit: FOOT)
  }
}
```

A GraphQL query with variables:

POST https://example.org/starwars/graphql
```graphql
query Hero($episode: Episode, $withFriends: Boolean!) {
  hero(episode: $episode) {
    name
    friends @include(if: $withFriends) {
      name
    }
  }
}

variables {
  "episode": "JEDI",
  "withFriends": false
}
```

GraphQL queries can also use Hurl templates.

Doc

Testing Response

Responses are optional, everything after HTTP is part of the response asserts.

# A request with (almost) no check:
GET https://foo.com

# A status code check:
GET https://foo.com
HTTP 200

# A test on response body
GET https://foo.com
HTTP 200
[Asserts]
jsonpath "$.state" == "running"

Testing Status Code

GET https://example.org/order/435
HTTP 200

Doc

GET https://example.org/order/435
# Testing status code is in a 200-300 range
HTTP *
[Asserts]
status >= 200
status < 300

Doc

Testing Response Headers

Use implicit response asserts to test header values:

GET https://example.org/index.html
HTTP 200
Set-Cookie: theme=light
Set-Cookie: sessionToken=abc123; Expires=Wed, 09 Jun 2021 10:18:14 GMT

Doc

Or use explicit response asserts with predicates:

GET https://example.org
HTTP 302
[Asserts]
header "Location" contains "www.example.net"

Doc

Implicit and explicit asserts can be combined:

GET https://example.org/index.html
HTTP 200
Set-Cookie: theme=light
Set-Cookie: sessionToken=abc123; Expires=Wed, 09 Jun 2021 10:18:14 GMT
[Asserts]
header "Location" contains "www.example.net"

Testing REST APIs

Asserting JSON body response (node values, collection count etc...) with JSONPath:

GET https://example.org/order
screencapability: low
HTTP 200
[Asserts]
jsonpath "$.validated" == true
jsonpath "$.userInfo.firstName" == "Franck"
jsonpath "$.userInfo.lastName" == "Herbert"
jsonpath "$.hasDevice" == false
jsonpath "$.links" count == 12
jsonpath "$.state" != null
jsonpath "$.order" matches "^order-\\d{8}$"
jsonpath "$.order" matches /^order-\d{8}$/     # Alternative syntax with regex literal
jsonpath "$.created" isIsoDate

Doc

Testing HTML Response

GET https://example.org
HTTP 200
Content-Type: text/html; charset=UTF-8
[Asserts]
xpath "string(/html/head/title)" contains "Example" # Check title
xpath "count(//p)" == 2  # Check the number of p
xpath "//p" count == 2  # Similar assert for p
xpath "boolean(count(//h2))" == false  # Check there is no h2  
xpath "//h2" not exists  # Similar assert for h2
xpath "string(//div[1])" matches /Hello.*/

Doc

Testing Set-Cookie Attributes

GET https://example.org/home
HTTP 200
[Asserts]
cookie "JSESSIONID" == "8400BAFE2F66443613DC38AE3D9D6239"
cookie "JSESSIONID[Value]" == "8400BAFE2F66443613DC38AE3D9D6239"
cookie "JSESSIONID[Expires]" contains "Wed, 13 Jan 2021"
cookie "JSESSIONID[Secure]" exists
cookie "JSESSIONID[HttpOnly]" exists
cookie "JSESSIONID[SameSite]" == "Lax"

Doc

Testing Bytes Content

Check the SHA-256 response body hash:

GET https://example.org/data.tar.gz
HTTP 200
[Asserts]
sha256 == hex,039058c6f2c0cb492c533b0a4d14ef77cc0f78abccced5287d84a1a2011cfb81;

Doc

SSL Certificate

Check the properties of a SSL certificate:

GET https://example.org
HTTP 200
[Asserts]
certificate "Subject" == "CN=example.org"
certificate "Issuer" == "C=US, O=Let's Encrypt, CN=R3"
certificate "Expire-Date" daysAfterNow > 15
certificate "Serial-Number" matches /[\da-f]+/

Doc

Checking Full Body

Use implicit body to test an exact JSON body match:

GET https://example.org/api/cats/123
HTTP 200
{
  "name" : "Purrsloud",
  "species" : "Cat",
  "favFoods" : ["wet food", "dry food", "<strong>any</strong> food"],
  "birthYear" : 2016,
  "photo" : "https://learnwebcode.github.io/json-example/images/cat-2.jpg"
}

Doc

Or an explicit assert file:

GET https://example.org/index.html
HTTP 200
[Asserts]
body == file,cat.json;

Doc

Implicit asserts supports XML body:

GET https://example.org/api/catalog
HTTP 200
<?xml version="1.0" encoding="UTF-8"?>
<catalog>
   <book id="bk101">
      <author>Gambardella, Matthew</author>
      <title>XML Developer's Guide</title>
      <genre>Computer</genre>
      <price>44.95</price>
      <publish_date>2000-10-01</publish_date>
      <description>An in-depth look at creating applications with XML.</description>
   </book>
</catalog>

Doc

Plain text:

GET https://example.org/models
HTTP 200
```
Year,Make,Model,Description,Price
1997,Ford,E350,"ac, abs, moon",3000.00
1999,Chevy,"Venture ""Extended Edition""","",4900.00
1999,Chevy,"Venture ""Extended Edition, Very Large""",,5000.00
1996,Jeep,Grand Cherokee,"MUST SELL! air, moon roof, loaded",4799.00
```

Doc

One line:

POST https://example.org/helloworld
HTTP 200
`Hello world!`

Doc

File:

GET https://example.org
HTTP 200
file,data.bin;

Doc

Reports

HTML Report

$ hurl --test --report-html build/report/ *.hurl

Doc

JUnit Report

$ hurl --test --report-junit build/report.xml *.hurl

Doc

TAP Report

$ hurl --test --report-tap build/report.txt *.hurl

Doc

JSON Output

A structured output of running Hurl files can be obtained with --json option. Each file will produce a JSON export of the run.

$ hurl --json *.hurl

Others

HTTP Version

Testing HTTP version (HTTP/1.0, HTTP/1.1, HTTP/2 or HTTP/3):

GET https://foo.com
HTTP/3 200

GET https://bar.com
HTTP/2 200

Doc

Polling and Retry

Retry request on any errors (asserts, captures, status code, runtime etc...):

# Create a new job
POST https://api.example.org/jobs
HTTP 201
[Captures]
job_id: jsonpath "$.id"
[Asserts]
jsonpath "$.state" == "RUNNING"


# Pull job status until it is completed
GET https://api.example.org/jobs/{{job_id}}
[Options]
retry: 10   # maximum number of retry, -1 for unlimited
HTTP 200
[Asserts]
jsonpath "$.state" == "COMPLETED"

Doc

Delaying Requests

Add delay for every request, or a particular requests:

# Delaying this request by 5s
GET https://example.org/turtle
[Options]
delay: 5000
HTTP 200

# No delay!
GET https://example.org/turtle
HTTP 200

Doc

Skipping Requests

# a, c, d are run, b is skipped
GET https://example.org/a

GET https://example.org/b
[Options]
skip: true

GET https://example.org/c

GET https://example.org/d

Doc

Testing Endpoint Performance

GET https://sample.org/helloworld
HTTP *
[Asserts]
duration < 1000   # Check that response time is less than one second

Doc

Using SOAP APIs

POST https://example.org/InStock
Content-Type: application/soap+xml; charset=utf-8
SOAPAction: "http://www.w3.org/2003/05/soap-envelope"
<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:m="https://example.org">
  <soap:Header></soap:Header>
  <soap:Body>
    <m:GetStockPrice>
      <m:StockName>GOOG</m:StockName>
    </m:GetStockPrice>
  </soap:Body>
</soap:Envelope>
HTTP 200

Doc

Capturing and Using a CSRF Token

GET https://example.org
HTTP 200
[Captures]
csrf_token: xpath "string(//meta[@name='_csrf_token']/@content)"


POST https://example.org/login?user=toto&password=1234
X-CSRF-TOKEN: {{csrf_token}}
HTTP 302

Doc

Checking Byte Order Mark (BOM) in Response Body

GET https://example.org/data.bin
HTTP 200
[Asserts]
bytes startsWith hex,efbbbf;

Doc

AWS Signature Version 4 Requests

Generate signed API requests with AWS Signature Version 4, as used by several cloud providers.

POST https://sts.eu-central-1.amazonaws.com/
[Options]
aws-sigv4: aws:amz:eu-central-1:sts
[FormParams]
Action: GetCallerIdentity
Version: 2011-06-15

The Access Key is given per --user, either with command line option or within the [Options] section:

POST https://sts.eu-central-1.amazonaws.com/
[Options]
aws-sigv4: aws:amz:eu-central-1:sts
user: bob=secret
[FormParams]
Action: GetCallerIdentity
Version: 2011-06-15

Doc

Manual

Name

hurl - run and test HTTP requests.

Synopsis

hurl [options] [FILE...]

Description

Hurl is a command line tool that runs HTTP requests defined in a simple plain text format.

It can chain requests, capture values and evaluate queries on headers and body response. Hurl is very versatile, it can be used for fetching data and testing HTTP sessions: HTML content, REST / SOAP / GraphQL APIs, or any other XML / JSON based APIs.

$ hurl session.hurl

If no input files are specified, input is read from stdin.

$ echo GET http://httpbin.org/get | hurl
    {
      "args": {},
      "headers": {
        "Accept": "*/*",
        "Accept-Encoding": "gzip",
        "Content-Length": "0",
        "Host": "httpbin.org",
        "User-Agent": "hurl/0.99.10",
        "X-Amzn-Trace-Id": "Root=1-5eedf4c7-520814d64e2f9249ea44e0"
      },
      "origin": "1.2.3.4",
      "url": "http://httpbin.org/get"
    }

Output goes to stdout by default. To have output go to a file, use the -o, --output option:

$ hurl -o output input.hurl

By default, Hurl executes all HTTP requests and outputs the response body of the last HTTP call.

To have a test oriented output, you can use --test option:

$ hurl --test *.hurl

Hurl File Format

The Hurl file format is fully documented in https://hurl.dev/docs/hurl-file.html

It consists of one or several HTTP requests

GET http://example.org/endpoint1
GET http://example.org/endpoint2

Capturing values

A value from an HTTP response can be-reused for successive HTTP requests.

A typical example occurs with CSRF tokens.

GET https://example.org
HTTP 200
# Capture the CSRF token value from html body.
[Captures]
csrf_token: xpath "normalize-space(//meta[@name='_csrf_token']/@content)"

# Do the login !
POST https://example.org/login?user=toto&password=1234
X-CSRF-TOKEN: {{csrf_token}}

More information on captures can be found here https://hurl.dev/docs/capturing-response.html

Asserts

The HTTP response defined in the Hurl file are used to make asserts. Responses are optional.

At the minimum, response includes assert on the HTTP status code.

GET http://example.org
HTTP 301

It can also include asserts on the response headers

GET http://example.org
HTTP 301
Location: http://www.example.org

Explicit asserts can be included by combining a query and a predicate

GET http://example.org
HTTP 301
[Asserts]
xpath "string(//title)" == "301 Moved"

With the addition of asserts, Hurl can be used as a testing tool to run scenarios.

More information on asserts can be found here https://hurl.dev/docs/asserting-response.html

Options

Options that exist in curl have exactly the same semantics.

Options specified on the command line are defined for every Hurl file's entry, except if they are tagged as cli-only (can not be defined in the Hurl request [Options] entry)

For instance:

$ hurl --location foo.hurl

will follow redirection for each entry in foo.hurl. You can also define an option only for a particular entry with an [Options] section. For instance, this Hurl file:

GET https://example.org
HTTP 301

GET https://example.org
[Options]
location: true
HTTP 200

will follow a redirection only for the second entry.

Option Description
--aws-sigv4 <PROVIDER1[:PROVIDER2[:REGION[:SERVICE]]]> Generate an Authorization header with an AWS SigV4 signature.

Use -u, --user to specify Access Key Id (username) and Secret Key (password).

To use temporary session credentials (e.g. for an AWS IAM Role), add the X-Amz-Security-Token header containing the session token.
--cacert <FILE> Specifies the certificate file for peer verification. The file may contain multiple CA certificates and must be in PEM format.
Normally Hurl is built to use a default file for this, so this option is typically used to alter that default file.
-E, --cert <CERTIFICATE[:PASSWORD]> Client certificate file and password.

See also --key.
--color Colorize debug output (the HTTP response output is not colorized).

This is a cli-only option.
--compressed Request a compressed response using one of the algorithms br, gzip, deflate and automatically decompress the content.
--connect-timeout <SECONDS> Maximum time in seconds that you allow Hurl's connection to take.

See also -m, --max-time.
--connect-to <HOST1:PORT1:HOST2:PORT2> For a request to the given HOST1:PORT1 pair, connect to HOST2:PORT2 instead. This option can be used several times in a command line.

See also --resolve.
--continue-on-error Continue executing requests to the end of the Hurl file even when an assert error occurs.
By default, Hurl exits after an assert error in the HTTP response.

Note that this option does not affect the behavior with multiple input Hurl files.

All the input files are executed independently. The result of one file does not affect the execution of the other Hurl files.

This is a cli-only option.
-b, --cookie <FILE> Read cookies from FILE (using the Netscape cookie file format).

Combined with -c, --cookie-jar, you can simulate a cookie storage between successive Hurl runs.

This is a cli-only option.
-c, --cookie-jar <FILE> Write cookies to FILE after running the session (only for one session).
The file will be written using the Netscape cookie file format.

Combined with -b, --cookie, you can simulate a cookie storage between successive Hurl runs.

This is a cli-only option.
--delay <MILLISECONDS> Sets delay before each request.
--error-format <FORMAT> Control the format of error message (short by default or long)

This is a cli-only option.
--file-root <DIR> Set root directory to import files in Hurl. This is used for files in multipart form data, request body and response output.
When it is not explicitly defined, files are relative to the Hurl file's directory.

This is a cli-only option.
--from-entry <ENTRY_NUMBER> Execute Hurl file from ENTRY_NUMBER (starting at 1).

This is a cli-only option.
--glob <GLOB> Specify input files that match the given glob pattern.

Multiple glob flags may be used. This flag supports common Unix glob patterns like *, ? and [].
However, to avoid your shell accidentally expanding glob patterns before Hurl handles them, you must use single quotes or double quotes around each pattern.

This is a cli-only option.
-0, --http1.0 Tells Hurl to use HTTP version 1.0 instead of using its internally preferred HTTP version.
--http1.1 Tells Hurl to use HTTP version 1.1.
--http2 Tells Hurl to use HTTP version 2.
For HTTPS, this means Hurl negotiates HTTP/2 in the TLS handshake. Hurl does this by default.
For HTTP, this means Hurl attempts to upgrade the request to HTTP/2 using the Upgrade: request header.
--http3 Tells Hurl to try HTTP/3 to the host in the URL, but fallback to earlier HTTP versions if the HTTP/3 connection establishment fails. HTTP/3 is only available for HTTPS and not for HTTP URLs.
--ignore-asserts Ignore all asserts defined in the Hurl file.

This is a cli-only option.
-i, --include Include the HTTP headers in the output

This is a cli-only option.
-k, --insecure This option explicitly allows Hurl to perform "insecure" SSL connections and transfers.
--interactive Stop between requests.

This is similar to a break point, You can then continue (Press C) or quit (Press Q).

This is a cli-only option.
-4, --ipv4 This option tells Hurl to use IPv4 addresses only when resolving host names, and not for example try IPv6.
-6, --ipv6 This option tells Hurl to use IPv6 addresses only when resolving host names, and not for example try IPv4.
--jobs <NUM> (Experimental) Maximum number of parallel jobs in parallel mode. Default value corresponds (in most cases) to the
current amount of CPUs.

See also --parallel.

This is a cli-only option.
--json Output each Hurl file result to JSON. The format is very closed to HAR format.

This is a cli-only option.
--key <KEY> Private key file name.
-L, --location Follow redirect. To limit the amount of redirects to follow use the --max-redirs option
--location-trusted Like -L, --location, but allows sending the name + password to all hosts that the site may redirect to.
This may or may not introduce a security breach if the site redirects you to a site to which you send your authentication info (which is plaintext in the case of HTTP Basic authentication).
--max-filesize <BYTES> Specify the maximum size (in bytes) of a file to download. If the file requested is larger than this value, the transfer does not start.

This is a cli-only option.
--max-redirs <NUM> Set maximum number of redirection-followings allowed

By default, the limit is set to 50 redirections. Set this option to -1 to make it unlimited.
-m, --max-time <SECONDS> Maximum time in seconds that you allow a request/response to take. This is the standard timeout.

See also --connect-timeout.

This is a cli-only option.
-n, --netrc Scan the .netrc file in the user's home directory for the username and password.

See also --netrc-file and --netrc-optional.
--netrc-file <FILE> Like --netrc, but provide the path to the netrc file.

See also --netrc-optional.
--netrc-optional Similar to --netrc, but make the .netrc usage optional.

See also --netrc-file.
--no-color Do not colorize output.

This is a cli-only option.
--no-output Suppress output. By default, Hurl outputs the body of the last response.

This is a cli-only option.
--noproxy <HOST(S)> Comma-separated list of hosts which do not use a proxy.

Override value from Environment variable no_proxy.
-o, --output <FILE> Write output to FILE instead of stdout.
--parallel (Experimental) Run files in parallel.

Each Hurl file is executed in its own worker thread, without sharing anything with the other workers. The default run mode is sequential.

See also --jobs.

This is a cli-only option.
--path-as-is Tell Hurl to not handle sequences of /../ or /./ in the given URL path. Normally Hurl will squash or merge them according to standards but with this option set you tell it not to do that.
-x, --proxy <[PROTOCOL://]HOST[:PORT]> Use the specified proxy.
--report-html <DIR> Generate HTML report in DIR.

If the HTML report already exists, it will be updated with the new test results.

This is a cli-only option.
--report-junit <FILE> Generate JUnit File.

If the FILE report already exists, it will be updated with the new test results.

This is a cli-only option.
--report-tap <FILE> Generate TAP report.

If the FILE report already exists, it will be updated with the new test results.

This is a cli-only option.
--resolve <HOST:PORT:ADDR> Provide a custom address for a specific host and port pair. Using this, you can make the Hurl requests(s) use a specified address and prevent the otherwise normally resolved address to be used. Consider it a sort of /etc/hosts alternative provided on the command line.
--retry <NUM> Maximum number of retries, 0 for no retries, -1 for unlimited retries. Retry happens if any error occurs (asserts, captures, runtimes etc...).
--retry-interval <MILLISECONDS> Duration in milliseconds between each retry. Default is 1000 ms.
--ssl-no-revoke (Windows) This option tells Hurl to disable certificate revocation checks. WARNING: this option loosens the SSL security, and by using this flag you ask for exactly that.

This is a cli-only option.
--test Activate test mode: with this, the HTTP response is not outputted anymore, progress is reported for each Hurl file tested, and a text summary is displayed when all files have been run.

This is a cli-only option.
--to-entry <ENTRY_NUMBER> Execute Hurl file to ENTRY_NUMBER (starting at 1).
Ignore the remaining of the file. It is useful for debugging a session.

This is a cli-only option.
--unix-socket <PATH> (HTTP) Connect through this Unix domain socket, instead of using the network.
-u, --user <USER:PASSWORD> Add basic Authentication header to each request.
-A, --user-agent <NAME> Specify the User-Agent string to send to the HTTP server.

This is a cli-only option.
--variable <NAME=VALUE> Define variable (name/value) to be used in Hurl templates.
--variables-file <FILE> Set properties file in which your define your variables.

Each variable is defined as name=value exactly as with --variable option.

Note that defining a variable twice produces an error.

This is a cli-only option.
-v, --verbose Turn on verbose output on standard error stream.
Useful for debugging.

A line starting with '>' means data sent by Hurl.
A line staring with '<' means data received by Hurl.
A line starting with '*' means additional info provided by Hurl.

If you only want HTTP headers in the output, -i, --include might be the option you're looking for.
--very-verbose Turn on more verbose output on standard error stream.

In contrast to --verbose option, this option outputs the full HTTP body request and response on standard error. In addition, lines starting with '**' are libcurl debug logs.
-h, --help Usage help. This lists all current command line options with a short description.
-V, --version Prints version information

Environment

Environment variables can only be specified in lowercase.

Using an environment variable to set the proxy has the same effect as using the -x, --proxy option.

Variable Description
http_proxy [PROTOCOL://]<HOST>[:PORT] Sets the proxy server to use for HTTP.
https_proxy [PROTOCOL://]<HOST>[:PORT] Sets the proxy server to use for HTTPS.
all_proxy [PROTOCOL://]<HOST>[:PORT] Sets the proxy server to use if no protocol-specific proxy is set.
no_proxy <comma-separated list of hosts> List of host names that shouldn't go through any proxy.
HURL_name value Define variable (name/value) to be used in Hurl templates. This is similar than --variable and --variables-file options.
NO_COLOR When set to a non-empty string, do not colorize output (see --no-color option).

Exit Codes

Value Description
0 Success.
1 Failed to parse command-line options.
2 Input File Parsing Error.
3 Runtime error (such as failure to connect to host).
4 Assert Error.

WWW

https://hurl.dev

See Also

curl(1) hurlfmt(1)

Installation

Binaries Installation

Linux

Precompiled binary is available at Hurl latest GitHub release:

$ INSTALL_DIR=/tmp
$ VERSION=4.3.0
$ curl --silent --location https://github.com/Orange-OpenSource/hurl/releases/download/$VERSION/hurl-$VERSION-x86_64-unknown-linux-gnu.tar.gz | tar xvz -C $INSTALL_DIR
$ export PATH=$INSTALL_DIR/hurl-$VERSION:$PATH

Debian / Ubuntu

For Debian / Ubuntu, Hurl can be installed using a binary .deb file provided in each Hurl release.

$ VERSION=4.3.0
$ curl --location --remote-name https://github.com/Orange-OpenSource/hurl/releases/download/$VERSION/hurl_${VERSION}_amd64.deb
$ sudo apt update && sudo apt install ./hurl_${VERSION}_amd64.deb

Alpine

Hurl is available on testing channel.

$ apk add --repository http://dl-cdn.alpinelinux.org/alpine/edge/testing hurl

Arch Linux / Manjaro

hurl-bin package for Arch Linux and derived distros is available via AUR.

NixOS / Nix

NixOS / Nix package is available on stable channel.

macOS

Precompiled binaries for Intel and ARM CPUs are available at Hurl latest GitHub release.

Homebrew

$ brew install hurl

MacPorts

$ sudo port install hurl

FreeBSD

$ sudo pkg install hurl

Windows

Zip File

Hurl can be installed from a standalone zip file at Hurl latest GitHub release. You will need to update your PATH variable.

Installer

An executable installer is also available at Hurl latest GitHub release.

Chocolatey

$ choco install hurl

Scoop

$ scoop install hurl

Windows Package Manager

$ winget install hurl

Cargo

If you're a Rust programmer, Hurl can be installed with cargo.

$ cargo install hurl

conda-forge

$ conda install -c conda-forge hurl

Hurl can also be installed with conda-forge powered package manager like pixi.

Docker

$ docker pull ghcr.io/orange-opensource/hurl:latest

npm

$ npm install --save-dev @orangeopensource/hurl

Building From Sources

Hurl sources are available in GitHub.

Build on Linux

Hurl depends on libssl, libcurl and libxml2 native libraries. You will need their development files in your platform.

Debian based distributions

$ apt install -y build-essential pkg-config libssl-dev libcurl4-openssl-dev libxml2-dev

Red Hat based distributions

$ yum install -y pkg-config gcc openssl-devel libxml2-devel

Arch based distributions

$ pacman -S --noconfirm pkgconf gcc glibc openssl libxml2

Alpine based distributions

$ apk add curl-dev gcc libxml2-dev musl-dev openssl-dev 

Build on macOS

$ xcode-select --install
$ brew install pkg-config

Hurl is written in Rust. You should install the latest stable release.

$ curl https://sh.rustup.rs -sSf | sh -s -- -y
$ source $HOME/.cargo/env
$ rustc --version
$ cargo --version

Then build hurl:

$ git clone https://github.com/Orange-OpenSource/hurl
$ cd hurl
$ cargo build --release
$ ./target/release/hurl --version

Build on Windows

Please follow the contrib on Windows section.

hurl's People

Contributors

actions-user avatar ad8lmondy avatar apparentorder avatar azzamsa avatar chenrui333 avatar cseeman avatar dependabot[bot] avatar dnsmichi avatar dorryspears avatar edg-l avatar elbart avatar fabricereix avatar guilherme-puida avatar hurl-bot avatar jcamiel avatar jleverenz avatar jsoref avatar lepapareil avatar lythenas avatar nikeee avatar pedromanse avatar ppaulweber avatar renovate-bot avatar roblillack avatar robozati avatar rosorio avatar teodorobert avatar thejus-paul avatar zklinger avatar zottelsheep avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hurl's Issues

Have a nice diff when comparing bodies

It looks like when using body assertions there is no simple way to compare actual and expected values.

E.g. : I make a request that returns body

{
    "id": 0,
    "name": "Frieda",
    "picture": "images/scottish-terrier.jpeg",
    "age": 3,
    "breed": "Scottish Terrier",
    "location": "Lisco, Alabama"
}

In my hurl file is as follows :

# Get a doggy thing:
GET https://example.net/api/dogs/

HTTP/1.1 200
{
    "id": 0,
    "name": "Julien",
    "picture": "images/scottish-terrier.jpeg",
    "age": 3,
    "breed": "Scottish Terrier",
    "location": "Lisco, Alabama"
}
<img width="1033" alt="Capture dโ€™eฬcran 2020-11-20 aฬ€ 14 55 40" src="https://user-images.githubusercontent.com/113336/99807944-77a2ee80-2b40-11eb-8855-ce559a130fcf.png">

The output is

error: Assert Body Value
  --> ./integration/test-files/nominal/edito/accessory-orange.hurl:36:1
   |
36 | {
   | ^ actual value is < {
    "id": 0,
    "name": "Frieda",
    "picture": "images/scottish-terrier.jpeg",
    "age": 3,
    "breed": "Scottish Terrier",
    "location": "Lisco, Alabama"
} >

Would it be possible to have a simple way to diff the actual and expected ? Like what is possible with JUnit failed string comparisons.

Capture dโ€™eฬcran 2020-11-20 aฬ€ 14 56 41

Add checksum body query (md5, sha1, sha256)

It's common to provide a checksum for HTTP resources to be verified.
It could be useful to add a checksum body query in asserts section.

A possible syntax:

GET https://sample.net/toto.tar.bz2

HTTP/1.1 200
[Asserts]
checksum "md5" "4b6fb79adc54152a9e5dc6aa3222b306"
GET https://github.com/Orange-OpenSource/hurl/releases/download/0.99.14/hurl-0.99.14-x86_64-osx.tar.gz

HTTP/1.1 200
[Asserts]
checksum "sha256" equals "b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66"

Maybe hash (vs checksum) could be a better query name :

GET https://github.com/Orange-OpenSource/hurl/releases/download/0.99.14/hurl-0.99.14-x86_64-osx.tar.gz

HTTP/1.1 200
[Asserts]
hash "sha256" equals "b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66"

A third possible syntax: just use the hash algo type

GET https://github.com/Orange-OpenSource/hurl/releases/download/0.99.14/hurl-0.99.14-x86_64-osx.tar.gz

HTTP/1.1 200
[Asserts]
sha256 equals "b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66"

So the possible syntax for the query:

  1. checksum where can be "sha1", "sha256", "md5"
  2. hash where can be "sha1", "sha256", "md5"
  3. 3 new queries sha1, sha256, md5

The two first has the advantage to be flexible for future algo, the last being more static and more succinct.

Note: the return type of the hash should be a string (lowercase, only 0-9, a-f), as there is no binary type in Hurl (for the moment).
So, while this asserts is syntactically correct, it could never be true: sha256 equals "zzz"

Request Cookies Section should not change cookie store

Cookies defined the request Cookies section should just add cookies in the request but should not change cookie storage.
This is consistent with curl.

For example: Send cookie1 in the request and cookie2 set from server endpoind

$ curl \
    --cookie cookie1=value1 \
    --cookie-jar /tmp/cookies \
    http://httpbin.org/cookies/set/name2/value2
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to target URL: <a href="/cookies">/cookies</a>. ..

$ cat /tmp/cookies 
# Netscape HTTP Cookie File
# https://curl.haxx.se/docs/http-cookies.html
# This file was generated by libcurl! Edit at your own risk.

httpbin.org	FALSE	/	FALSE	0	name2	value2

with the current hurl

$ cat <<END | hurl --cookie-jar /tmp/cookies
GET http://httpbin.org/cookies/set/cookie2/value2
[Cookies]
cookie1: value1
END

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to target URL: <a href="/cookies">/cookies</a>...

$ cat /tmp/cookies 
# Netscape HTTP Cookie File
# This file was generated by hurl

httpbin.org	FALSE	/	FALSE	0	cookie1	value1
httpbin.org	FALSE	/	FALSE	0	cookie2	value2

Assert of response cookies

The current query cookie return the cookie value (and ignores the other attributes such as path, domain, ...)
cookie "theme" equals "light"

It would be interesting to fully support cookies as defined in the http spec.

the query cookie could return a Cookie Object instead of a simple string.
This object would include a mandatory string value, an optional domain, an option value, ...

The predicate equals (and matches?) could then be applied on this query taking a String value.
There are several options for the semantic of this String value.

For example

cookie "sessionToken" equals "abc123; Expires=Wed, 09 Jun 2021 10:18:14 GMT"
the path and domain must not be defined in the response cookie

cookie "LSID" equals "DQAAAKโ€ฆEaem_vYg; Path=/accounts; Expires=Wed, 13 Jan 2021 22:23:01 GMT; Secure; HttpOnly"
the order of attributes does not matter

cookie "LSID" matches "DQAAAKโ€ฆEaem_vYg; Path=/accounts; Expires=.*; Secure; HttpOnly"
when you need to use a regex you explicitly use the matched predicate

instead it could be easier to omit the attribute to ignore the attribute.
cookie "LSID" equals "DQAAAKโ€ฆEaem_vYg; Path=/accounts; Secure; HttpOnly"
you don't check the Expires attribute at all. It does not matter whether it is present on not.
But then, how do you check whether a cookie is a session cookie?


Another option: ยดcookieยด could be extended as a query expression like jsonpath/xpath:

Check value:
cookie "toto" equals "tata"

Check expires:
cookie "toto[expires]" equals "Wed, 09 Jun 2021 10:18:14 GMT"
cookie "toto[expires]" exist

Check Path:
cookie "toto[path]" matches "/test.*"

We can borrow XPath syntax for attribute:

cookie "toto[@path]" equals "/domain"

Or we can create another type of query ยดcookiepathโ€™ and keep the actual ยดcookieยด query.


Possible query syntax:

  • cookie "toto[expires]" exist
  • cookie "toto[@expires]" exist
  • cookie "toto.expires" exist
  • cookie "toto@expires" exist

=> cookie "toto[Expires]" exist is accepted as the syntax

Possible attributes:
Expires=
Max-Age=
Domain=
Path=
Secure
HttpOnly
SameSite=Strict
SameSite=Lax
SameSite=None

cookie "toto[max-age]" exist
cookie "toto[Max-Age]" exist equivalent


Template variable should probably also be supported for the cookie name.
cookie "{{cookie_name}}[Expired]"


Definition of queries in the grammar

xpath-query 	= "xpath" sp sp* quoted-string
cookie-query 	= "cookie" sp sp* quoted-string

contrary to xpath, the cookie-query is both specific to hurl and very easy.
it could be fully described as follow

cookie-query 	= "cookie" sp sp* """  cookie-query-expr  """
cookie-query-expr = cookie-query-name ( "[" sp* cookie-query-attribute sp* "]" )
cookie-query-name = <[a-zA-Z0-9{}]+>
cookie-query-attribute = Value | value
                       | Expires | expires
                       | Max-Age | max-age
                       | Domain | domain
                       | Path | path
                       | Secure | secure
                       | HttpOnly | httponly
                       | SameSite | samesite

The parsing implementation can choose to be a bit stricter than the grammar

for example,
rejecting escape character
cookie "toto\u{5B}Expires]"

accepting variable only for the cookie name
cookie "toto[{{field}}]"

Predicates with not qualifier

The not qualifier in the predicate negates most of the time the predicate result.

But there are a few cases, for which it does not invert a predicate failure.
Hurl detects some inconsistencies between the predicate and the value to which it is applied and prefer to make the assert fail.

Example 1 - the value (returned by the query) is 1 (integer)

All the following predicates fail:

equals 2
equals null
contains "toto"

Negating the predicate does not necessarily make the predicate succeeds.

not equals 2       
not equals null  
not contains "toto" => failure

The last one fails because the contains predicate only apply to String value, and thus it does not make sense to apply it to a integer value.
This is equivalent to a type error.

Example 2 - the value is not found

All the following predicates fail:

exist
equals 1
contains "toto"

all these predicates will success if there are negated

not exist
not equals 1
not contains "toto"

There can't be any "type checking" when no value is returned.

Example 3 - the value is present but does not have a value

for example, the Secure cookie attribute, it is present or not
but it does not have a value

All the following predicates fail:

equals true
equals false
equals 1
contains "toto"

All these predicates will fail even if they are negated.

not equals true
not equals false
not equals 1
not contains "toto"

They must be applied to a value.
exist is the only predicate that can be applied.

Hurl usage doesn't end with newline

When typing hurl + enter, Hurl's usage is shown. It should end with a newline.
Actual:

~/Documents/Dev/hurl-jvm $ hurl
hurl 0.99.13
Run hurl FILE(s) or standard input

USAGE:
    hurl [OPTIONS] [--] [INPUT]...

OPTIONS:
        --append                              Append sessions to json output
        --color                               Colorize Output
        --compressed                          Request compressed response (using deflate or gzip)
        --connect-timeout <SECONDS>           Maximum time allowed for connection
    -b, --cookie <FILE>                       Read cookies from FILE
    -c, --cookie-jar <FILE>                   Write cookies to FILE after running the session (only for one session)
...
        --variable <NAME=VALUE>...            Define a variable
    -v, --verbose                             Turn on verbose output
    -h, --help                                Prints help information
    -V, --version                             Prints version information

ARGS:
    <INPUT>...    Sets the input file to use~/Documents/Dev/hurl-jvm $ 

Expected:

~/Documents/Dev/hurl-jvm $ hurl
hurl 0.99.13
Run hurl FILE(s) or standard input

USAGE:
    hurl [OPTIONS] [--] [INPUT]...

OPTIONS:
        --append                              Append sessions to json output
        --color                               Colorize Output
        --compressed                          Request compressed response (using deflate or gzip)
        --connect-timeout <SECONDS>           Maximum time allowed for connection
    -b, --cookie <FILE>                       Read cookies from FILE
    -c, --cookie-jar <FILE>                   Write cookies to FILE after running the session (only for one session)
...
        --variable <NAME=VALUE>...            Define a variable
    -v, --verbose                             Turn on verbose output
    -h, --help                                Prints help information
    -V, --version                             Prints version information

ARGS:
    <INPUT>...    Sets the input file to use
~/Documents/Dev/hurl-jvm $ 

Add option --compressed

Add curl option --compressed:

  • add header request 'Accept-Encoding'
  • automatically decompress the body of the last response (if this one has been compressed)

Note that the body of all the other responses won't be decompressed.

Add Hurl File JSON export

Add an option (with hurlfmt ?) to export an Hurl file (the last) to json, to provide an agnostic file format.

Add option --interactive

Stop between each request.
Map commands to keys.

Start by implemented the following commands:

  • Key Q (Quit)
  • key C (Continue)

The documentation is not really clear about header assertions

Hi,

I have a test with :

GET http://someUrl
Accept-Encoding: gzip

HTTP/1.1 200
Content-Encoding: gzip

My server actually returns :

Content-Encoding: gzip
Content-Encoding: gzip

Which is a bug, but that's another story...

The assertion passes, and maybe it's OK.

But when I change the assertion to :

GET http://someUrl
Accept-Encoding: gzip

HTTP/1.1 200
Content-Encoding: "gzip"

It fails with

    |
 21 | Content-Encoding: "gzip"
    |                   ^^^^^^ actual value is <["gzip", "gzip"]>

Do you think that it is logical ? There is no mention of Header: "expectedValue", only Header: expectedValue. I mean is the Header: expectedValue syntax just a contains() ?

Maybe just a clarification could be added...

Thanks

Can not parse user in url (Basic Authentication)

Hurl should support a user defined within a url for basic authentication

$ echo GET http://bob:secret@localhost:8000/basic-authentication | hurl
error: Parsing literal
  --> -:1:22
   |
 1 | GET http://bob:secret@localhost:8000/basic-authentication
   |                      ^ expecting 'line_terminator'
   |

Explain --json, --append --html interaction

Let's say we have file1.hurl, file2.hurl file3.hurl.

To get a json view of all files:

hurl --json /tmp/files.json file1.hurl file2.hurl file3.hurl

=> /tmp/files.json does represent the 3 files

hurl --json /tmp/files.json --append file1.hurl
hurl --json /tmp/files.json --append file2.hurl
hurl --json /tmp/files.json --append file3.hurl

=> /tmp/files.json does represent the 3 files

To get a html view of all files:

hurl --html /tmp/ file1.hurl file2.hurl file3.hurl

=> /tmp/*.html does represent the 3 files

hurl --json /tmp/files.json --append --html /tmp/ file1.hurl
hurl --json /tmp/files.json --append --html /tmp/ file2.hurl
hurl --json /tmp/files.json --append --html /tmp/ file3.hurl

=> /tmp/*.html does represent the 3 files

Hurl MVP1

MVP1 - End October

  • Packaging .deb files for release (at least Ubuntu, Debian) (add dependencies on libcurl4)
  • Add json export in binary
  • Update doc with json export
  • Update doc with curl
  • Fix cookies section managment
  • Dog fooding libcurl integration
  • Add timeout options
  • Update timeout option doc
  • Update CookiePath doc
  • Testing gzip response

MVP2

  • Interractive session
  • Add Cookie Storage directives
  • libcurl3

When exporting greater and less predicates in JSON, value is exported as string

When exporting greater and less predicates in JSON, value is exported as string.

integration/tests/assert_json.hurl :

GET http://localhost:8000/assert-json
HTTP/1.0 200
[Asserts]
...
jsonpath "$.count" greaterThan 1

In integration/tests/assert_json.json:

{
  "query": {
    "type": "jsonpath",
    "expr": "$.count"
  },
  "predicate": {
    "type": "greater-than",
    "value": "1"
  }
}

Feature Requests: Imports

First of all: thanks for building this! I really like the general idea.

A great feature would be imports to reuse requests (e.g. login flows) and to be able to scale larger test suites.

Update hurlfmt usage

use --output option with value (text, json, html)
instead of dedicated option --json and --html

Typo in greater than and less than JSON export

When exporting "greater than" and "less than" predicates, the value for type in the JSON export should reflect the grammar naming;

For example:
"less than" predicate is defined as less-predicate in Hurl grammar and should be export as {"type":"less"...}

Support wildcard value in implicit status code reponse

Wildcard * can be used in HTTP version to disable any check on the HTTP version:

GET http://localhost:8000/timeout
HTTP/* 200

It could be useful to also support wildcard for status code.
For instance:

GET http://localhost:8000/timeout
HTTP/* *
[Asserts]
status not equals 200

Status code is explicitly tested not to be equals to 200

Allow cargo install

Currently only cargo build is used in the CI to build hurl executable
Update your Cargo.toml so that you can also run cargo install

$ cargo install
error: found a virtual manifest at `...hurl/Cargo.toml` instead of a package manifest

Add comparison predicates

add the 4 comparison predicates:

  • greaterThan
  • greaterThanOrEquals
  • lessThan
  • lessThanOrEquals

They can all take an Integer or a float param as defined in the grammar

Add option --variables-file / --variables

Currently, variables can be set at the command-line with the option --variable <name=value>

It can become a bit tedious when the number of variables increases.
Hurl could also read variables from an input properties file with an option like --variables-file <FILE>

The values defined in the properties will be exactly the same as the one define din the command-line.

Improving Assert Error for String

Here is a typical assert error in hurl.

error: Assert Failure
  --> tests/error_assert_value_error.hurl:8:0
   |
 8 | jsonpath "$.count" greaterThan 5
   |   actual:   int <2>
   |   expected: greater than int <5>
   |

This error is quite clear, but for string comparison, it might not be as clear.

For example, for multiline string, you may receive the following error

error: Assert Failure
  --> test.hurl:4:0
   |
 4 | body equals "one\ntwo\nthree"
   |   actual:   string <one
   |   to
   |   three>
   |   expected: string <one
   |   two
   |   three>
   |

It could be better to display both the actual and expected strings on one line.
Converting the newline character to \n

The previous error would become:

error: Assert Failure
  --> test.hurl:4:0
   |
 4 | body equals "one\ntwo\nthree"
   |   actual:   string <one\nto\nthree>
   |   expected: string <one\ntwo\nthree>
   |

To go one step further, the different characters could be highlighted (for example in red)
one\ntwo\nthree

Add option -u, --user <user:password>

Add the -user option from curl
in order to specify the user name and password to use for server authentication.

This is useful when querying an API for which you need to provide an Authorization header for each request.

Serialization of cookie query for Expires attributes with hurlfmt

When exporting an Hurl file containing a cookie query with Expires attributes, the serialized attribute is Expire:

GET http://localhost

HTTP/1.1 200
[Asserts]
cookie "Session[Expires]" exists
$ hurlfmt --format json /tmp/test.hurl
{
  "entries": [
    {
      "request": {
        "method": "GET",
        "url": "http://localhost"
      },
      "response": {
        "version": "HTTP/1.1",
        "status": 200,
        "asserts": [
          {
            "query": {
              "type": "cookie",
              "expr": "Session[Expire]"
            },
            "predicate": {
              "type": "exist"
            }
          }
        ]
      }
    }
  ]
}

Type input variables

Values used in Hurl in Captures/Assert are typed (integer, boolean, string,...)
On the opposite, the input variables (from the option --variable) are all strings.

sample.hurl 
 1  GET http://localhost:8000/hello
 2  HTTP/* 200
 3  [Asserts]
 4  variable "count" equals "1"
 5  variable "count" equals 1

$ hurl --variable count=1 sample.hurl 
error: Assert Failure
  --> /tmp/sample.hurl:5:0
   |
 5 | variable "count" equals 1
   |   actual:   string <1>
   |   expected: int <1>
   |

It will be interesting to type the input variable implicitly.
The string <1> should be treated as integer, therefore the first assert should fail rather than the second.

Similarly, the value should be treated as bool.
To force a string value, the value can be quoted.
This is consistent with the predicate value.

For example, the id <123> is passed as string as follow:

hurl --variable id='"123"' sample.hurl 

Do not add header Expect automatically

by default, curl might add the Expect: 100-continue header automatically, for example for a PUT request.

In Hurl, the header will never be sent implicitly.
It would have to be added explicitly in the request header.

MultipartFormData is not present in json export

MultipartFormData is not present in json export.

With this file:

POST http://localhost:8000/multipart-form-data
[MultipartFormData]
key1: value1
upload1: file,hello.txt;
upload2: file,hello.html;
upload3: file,hello.txt; text/html

HTTP/1.0 200

and

hurl --json /tmp/test.json -v integration/tests/multipart_form_data.hurl

The report is :

[
  {
    "filename": "integration/tests/multipart_form_data.hurl",
    "entries": [
      {
        "request": {
          "method": "POST",
          "url": "http://localhost:8000/multipart-form-data",
          "queryString": [],
          "headers": [],
          "cookies": [],
          "body": ""
        },
        "response": {
          "httpVersion": "HTTP/1.0",
          "status": 200,
          "cookies": [],
          "headers": [
            {
              "name": "Content-Type",
              "value": "text/html; charset=utf-8"
            },
            {
              "name": "Content-Length",
              "value": "0"
            },
            {
              "name": "Server",
              "value": "Werkzeug/1.0.1 Python/3.8.5"
            },
            {
              "name": "Date",
              "value": "Mon, 02 Nov 2020 13:00:52 GMT"
            }
          ]
        },
        "captures": [],
        "asserts": [
          {
            "actual": "1.0",
            "expected": "1.0"
          },
          {}
        ],
        "time": 9
      }
    ],
    "success": true,
    "time": 9,
    "cookies": []
  }
]

add the --parallel option to run *.hurl files in parallel instead of sequentially

Hi guys ๐Ÿ˜„

Currently when I launch this command

hurl a.hurl b.hurl c.hurl

hurl runs each *.hurl file sequentially, first a.hurl, then b.hurl, then c.hurl.

# tests started sequentially
hurl a.hurl b.hurl c.hurl

โ”€ BEGIN
       โ”œโ”€ a.hurl
                โ”œโ”€ b.hurl
                         โ”œโ”€ c.hurl
โ”€ END

I propose you to add the --parallel option allowing hurl to run *.hurl files in parallel:

# all tests started at same time
hurl --parallel a.hurl b.hurl c.hurl

โ”€ BEGIN
      โ”œโ”€ a.hurl โ”œโ”€ b.hurl โ”œโ”€ c.hurl
โ”€ END
# tests started two at a time
hurl --parallel 2 a.hurl b.hurl c.hurl

โ”€ BEGIN
       โ”œโ”€ a.hurl  โ”œโ”€ b.hurl
       โ”œโ”€ c.hurl
โ”€ END

What do you think about it?

add the --concurrency option to launch multiple runs of *.hurl files instead of one

Hi guys ๐Ÿ˜„

In the context of our project we see two huge advantages using hurl :

  • It is an ideal solution for functionally testing all the page sequences what we call customer paths on our web application. This allows us to guarantee the non-regression between versions.
  • The simplicity of its language allows non-developers to generate a multitude of scenarios easily.

However, we cannot "yet" use hurl for our metrology tests, forcing us to duplicate all our scenarios in other tools that are less accessible to non-developers... ๐Ÿ˜ข

Currently when I launch this command

hurl a.hurl b.hurl c.hurl

hurl runs each *.hurl file one time, first a.hurl, then b.hurl, then c.hurl.

โ”€ BEGIN
       โ”œโ”€ a.hurl
                โ”œโ”€ b.hurl
                         โ”œโ”€ c.hurl
โ”€ END

I propose you to add the --concurrency option allowing hurl to launch multiple runs of each *.hurl file at a time, for example:

hurl --concurrency 3 a.hurl b.hurl c.hurl
โ”€ BEGIN
       โ”œโ”€ a.hurl โ”œโ”€ a.hurl โ”œโ”€ a.hurl
                                    โ”œโ”€ b.hurl โ”œโ”€ b.hurl โ”œโ”€ b.hurl
                                                                 โ”œโ”€ c.hurl โ”œโ”€ c.hurl โ”œโ”€ c.hurl
โ”€ END

What do you think about it ?

Improving Assert Error for a given textual body

Response body can be tested directly without explicit asserts as follow:

test.hurl
1  GET http://example.com/text
2  HTTP/1.1 200
3  ```
4  one
5  two
6  three
7  ```

If the server returns the following response

one 
to
three

the following error will be produced:

error: Assert Body Value
  --> test.hurl:4:1
   |
 4 | one
   | ^ actual value is <one
   |   to
   |   three>

Instead of taking into account the full multiline body, we could focus on the first line that differs.

The previous errors would become:

error: Assert Body Value
  --> test.hurl:5:1
   |
 5 | two
   | ^ actual value is <to>
   |   

This fits quite well with the line-oriented structure of hurl files.

Testing json output can also be a good example.

test.hurl
1  GET http://example.com/json
2  HTTP/1.1 200
3  {
4     "name": "Bob",
5     "age": 32
6  }

If the server returns the following response

{
   "name": "Bob",
   "age": 30
}

The following error will be produced:

error: Assert Body Value
  --> test.hurl:3:1
  |
3 | {
  | ^ actual value is <{
  |       "user": "Bob"
  |       "age": "30"
  |   }>

Taking into account only the first line in error, we get the following error instead:

error: Assert Body Value
  --> test.hurl:5:1
   |
 5 |    "age": 32
   |  ^ actual value is <    "age": 30>
   |   

We could even position the error at the exact different chracter

error: Assert Body Value
  --> test.hurl:5:12
   |
 5 |    "age": 32
   |            ^ actual value is <0>
   |   

Do you plan to support time-based assertions ?

For some use-cases we would like to check that the http response has been returned within x milliseconds, do you think it has a chance to be implemented ? Maybe we could think about the response payload size also (if Content-Length header is not set correctly)

What are your thoughts ?

Runner directives / Cookies Section behavior

Currently, setting a cookie via the cookie section updates the cookiejar
in order to send a cookie without updated the cookiejar, an explicit header 'Cookie' must be used

In the future, we should be able to interact with the runner in a hurl file with predefined directives
(such as clear cookie storage)

Arithmetic predicates for number (Integer or Float)

Predicates with numbers such as (equals, greaterThan, ...) are "arithmetic".

For example:
for the following json response

{
    "count": 5,
    "total": 50.0
}

All the following asserts are valid

jsonpath "$.count" equals 5
jsonpath "$.count" equals 5.0
jsonpath "$.count" greaterThan 1
jsonpath "$.count" greaterThan 1.0
jsonpath "$.total" equals 50
jsonpath "$.total" equals 50.0
jsonpath "$.total" greaterThan 0.2
jsonpath "$.total" greaterThan 0

The value type can be checked with an additional predicate

jsonpath "$.count" isInteger
jsonpath "$.total" isFloat

Additional:

The intent for the following assert might not be clear:
jsonpath "$.count" equals 5.0
Do we expect a float value?

In this case, Hurl could emit a warning like:
"You expected value should be replaced by an integer value"

Uncompress response body for textual queries

Textual queries such as xpath and jsonpath assume that the response body is text.
If the response body has been compressed (protocol given in response header Content-Encoding), it must be decompressed automatically.

Improving pattern for regex capture and matches predicates

Currently, regex patterns are used in :

  • regex query
  • matches predicate
GET http://localhost:8000/assert-regex

HTTP/1.0 200
[Asserts]
regex "Hello ([0-9]+)!" not exists
regex "Hello ([a-zA-Z]+)!" equals "World"
body matches "Hello [a-zA-Z]+!"

Pattern in Hurl grammar use quoted-string: the problem is that backslash in quoted-string should be escaped. So to use special character in pattern (like \d, \s), backslash should be escaped, degrading the readability of the pattern:

regex "Hello (\\d)!" not exists

In Rust, Python, Kotlin, raw strings are used to improve the regex readability. In Rust, for instance:

use regex::Regex;
let re = Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap();
assert!(re.is_match("2014-01-01"));

In the Hurl grammar, we already have raw string (```something\ntutu``` is literally the string "something\ntutu" without a newline between something and tutu). We could reuse this syntax when using regex query or matches predicates:

Before:

GET http://localhost:8000/assert-regex

HTTP/1.0 200
[Asserts]
regex "abc(\\d+)!" exists
jsonpath "$.date1" matches "\\d{4}-\\d{2}-\\d{2}"

After:

GET http://localhost:8000/assert-regex

HTTP/1.0 200
[Asserts]
regex ```abc(\d+)!``` exists
jsonpath "$.date1" matches ```\d{4}-\d{2}-\d{2}```

We could also introduce another syntax, like Python raw string and Rust raw string and keep ``` for body raw string:

GET http://localhost:8000/assert-regex

HTTP/1.0 200
[Asserts]
regex r"abc(\d+)!" exists
jsonpath "$.date1" matches r"\d{4}-\d{2}-\d{2}"

raw-string like r"abcd" could be used anywhere a quoted-string is used (not limited to regex capture and matches predicate)

Valid Jsonpath query is not parsed

the jsonpath query $.[0].name should be valid
though this is probably better written as $[0].name

With the following json response body

[
  { "id": 1, "name": "Bob"},
  { "id": 2, "name": "Bill"}
]

it should return "Bob"

Improve Template Support in JSON body

Currently, templates in json body is supported only for string

{
  "name": "{{name}}",
  "age": 30
}

the following file is not supported

{
  "name": "{{name}}",
  "age": {{age}}
}

Note that It is still possible to send this json body with a raw string.
But the content is not validated as json.

Default ContentType on Multipart Form Data

Hurl seems to add a default content type for file form datas:

POST http://localhost:8000/multipart-form-data
[MultipartFormData]
key1: value1
upload1: file,hello.txt;
upload2: file,hello.html; text/html
upload3: file,hello.txt; text/plain

HTTP/1.0 200

upload1 file is sent with content type "text/plain" whereas no content type has been specified.

There are two options:

Each part MAY have an (optional) "Content-Type" header field, which
defaults to "text/plain". If the contents of a file are to be sent,
the file data SHOULD be labeled with an appropriate media type, if
known, or "application/octet-stream".

So in this case, upload1 content type should be "application/octet-stream"

  • option 2: when no content type is specified, add a content type base on the file extension.

Curl seems to use option 2:

const char *Curl_mime_contenttype(const char *filename)
{
  /*
   * If no content type was specified, we scan through a few well-known
   * extensions and pick the first we match!
   */
  struct ContentType {
    const char *extension;
    const char *type;
  };
  static const struct ContentType ctts[] = {
    {".gif",  "image/gif"},
    {".jpg",  "image/jpeg"},
    {".jpeg", "image/jpeg"},
    {".png",  "image/png"},
    {".svg",  "image/svg+xml"},
    {".txt",  "text/plain"},
    {".htm",  "text/html"},
    {".html", "text/html"},
    {".pdf",  "application/pdf"},
    {".xml",  "application/xml"}
  };

But I prefer option1 => no implicit.
In any case, we should update the doc to explain the behaviour.

Add predicates to test value types

Add the predicates isInteger, isFloat, isBoolean, isString, isCollection

for example, for the following json response

{
    "count": 4,
}

The following asserts fails

jsonpath "$.count" isString
jsonpath "$.count" isFloat

while the following assert is successful
jsonpath "$.count" isInteger

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.