Giter VIP home page Giter VIP logo

teakozi's Introduction

npm version Build Status

Teakozi

Teakozi is a declarative REST API testing framework for testing micro-services. Here are the capabilities.

  • Writing tests as readable YML files
  • Externalize the configuration / environment specific parameters
  • Dynamic Data - Collect Property and use it in subsequent steps.
  • Scenario based testing. Externalize the test data and validation data from test specification
  • Reuse API calls in multiple tests
  • Validate response schema as per Swagger definitions
  • Library extensions

How to use Teakozi

  • include teakozi in your node application
$ npm install teakozi

create a directory in your project consisting of the following subdirectories

  • tests
  • tests/[testname].yml
  • modules
  • modules/[modulename].yml
  • models
  • models/[modelxx].js
  • config
  • config/index.js
  • payload
  • payload/[xxx].json
Directory Purpose
tests contains the yml files that describe the tests. You can arrange the test yml files in any suitable folder hiererchy. the framework will get all yml files in directory under this tree and ignore all the non .yml files
modules reusable test steps that can be refered in the test yml file
models externalize the data to drive the tests
config just one index.js file
payload the body of the content that can be called in a post step. This folder also contains the json payload against which you need to validate your responses.

config/index.js

This file contains

  1. key value for replacing in the test yml files.
  2. The index file can optionally have a tag called swagger pointing to the location of the swagger file from project home directory.
  3. Also has extension functions that can be defined and used in the test definition. Take a look at section library functions below.
  4. sync flag - whether the test should run sync / async. please refer to section sync / async as the behavior is dependent on test sync attribute and config sync attribute

see the example folder in the repo for reference. you need to update the config/index.js with your github auth to

Externalize data from tests

Teakozi allows you to externalize the data from test definition

Reusable modules

you can refer to the steps in the module as - {{module name}}

Configuration data

{{xxx}} marked content gets replaced from the configuration file keys-> value

Dynamic data

you can collect data in a step and reuse in the subsequent steps. so you can collect title and refer it in subsequent steps like ~title~

Library Functions

Add library functions in your config/index.js file as follows

Sync / Async calling

sync tag can be applied at the config level and also at file level. the framework will execute the test in the following manner

SYNC Config true Config false Config undefined
Test - true Sync Sync Sync
Test- false Sync Async Async
Test - undefined Sync Async Sync

This behavior is subject to change based on user feeback. Current phillosophy - default to sync when not explicit

module.exports = {
  swagger:'/config/swagger.yml',
  lib:{
    increment : function(i){return ++i},
    decrement : function(i){return --i}
  }
};

you can now use the increment and decrement functions while collecting the parameters from response. example

    collect:
     incremeted_id: increment->$.order[0].id
    print:
     - $.order[0].id
     - incremeted_id

Calling the tests from your node app

in a .js file call

require("teakozi").start("project/example")

Structure of a test yml file (tests directory)

Property required type Purpose
name required string Name of the Test Case
tags optional string comma saperated list of tags for the test. you can use thse tags to filter / select the test you would like to take for a test run
sync optional bool Whether the test should run sync / async. please refer to section sync / async as the behavior is dependent on this and config sync flag
iterate optional string Name of the module file that returns an array of objects. The test steps defined will be repeated for all the elements of this array
steps required array See Steps Section

Steps

Property required type Purpose
get / post / put / delete / local / mongo required object method to be called
name required string Name of the Test Step
delay optional int[,int] Delay in seconds to start this step. Optionally separated by comma, you can specify how may times should you loop before giving up. Example 3,5 means do the step after 3 seconds delay. if fails then retry for 5 times every 3 seconds. if passing move to next step
iterate optional string name of the module that returns array of objects. the call will get repeated for each of the members of the returned array
check required object what to do with the response received. What asserts to do and what properties to pick from the response to be used in subsequent calls
collect optional object the jsonpaths values that should be collected for use in subsequent calls.
print optional array the jsonpaths values that should be printed on the console output for debugging purposes
save optional object the jsonpaths values that should be saved and the file name where it should be saved example : $: my_response_content The files are saved in the generated folder on the project
skip_on_error optional bool Default: true. If entered false, it will execute the step even if the previous steps have failed. Can be used to do cleanup / teardown activities

get / post / put / delete

Property required type Purpose
url required string The API to be called
json na -get, optional - for post, optional - put, na - delete string the name of file in modules that returns the json to be posted
file na -get, optional - for post, optional - put, na - delete string Name of the file in modules (plain text) that needs to be posted
override na -get, optional - for post, optional - put, na - delete array jsonpaths that need to be updated with value
headers optional -get, optional - for post, optional - put, optional - delete array headers that go on the http requests. like Authorization content-type etc

example:

post:
     url: "https://api.github.com/gists/~gist_id~/comments"
     json: "create_comment.json"
     override:
      $.body: ~comment_text~
     headers:
      Authorization: Basic xxxx
      content-type: application/json
      User-Agent: Teakozi-test

local

Property required type Purpose
file required string name of the file in the payload folder

Example:

local:
    file: roads

mongo

Property required type Purpose
server required string connection string / endpoint of the mongo server like "mongodb://localhost:27017"
db required string the name of the database
collection required string collection name in the above db from where the records have to be fetched
query required object the filter criteria of querying the records from collection

Example:

  - mongo:
     server: "mongodb://localhost:27017"
     db: 'test'
     collection: "test"
     query:
       name: j

collect

Collects the propoerties from the payload response that would be used in the subsequent steps

    collect:
     title: $..[0].name
     project_id: $..[0].id

collect also supports reading complete or subset of json recieved like

    collect:
     all_roads: $..Name

this will collect all the Name fields comming at any part of the json. using JSON path expression on the right hand side you can collect specific sections of json for comparision

print

Enables debugging of the call by printing the jsonpaths from the response

  - get:
     url: "http://localhost:3060/k.json"
    name: "Get Auth K"
    print:
     - status
     - $.mykey
    check:
     status: 200
    collect:
     auth_key: $.mykey

print also supports checking the objects / subset of json as defined by json path like

print:
 - status # status of the header
 - $..Name # All names appearing in the json
 - $[0].Name # Prints name value of the first record
 - $ # Prints complete JSON

check

Property required type Purpose
status optional number what status code is expected in response
schema optional string you can add the name of the Model corresponding to the expected response. The swagger definition should be done in config/index.js
eq optional object the jsonpaths that should be equal to
neq optional object the jsonpaths that should notbe equal to
null optional array the jsonpaths that should be null
deepEqual optional array The jsonpaths and what collected objects to check against.
shallowEqual optional array compare two collected json to match and validate such that each element of A is present in the B (A is a subset B is a superset)
regex optional array The jsonpaths and their matching regex. The regex expression can be external mentioned as regex_for_currency or mentioned inline

example:

    check:
     status: 200
     schema: Domain
     body:
      eq:
       $.length: 20
       $..[0].name: "devops_best_practices"
      regex:
       $.domain: "^Ji[a-z]*"
      neq:
       $.length: 0
      null:
       - $.nonexistant
       - $.someother_nonexistant
      deepEqual:
        $: full_roads_payload
        $..Name: all_roads
      shallowEqual:
        - sub: sub
          super: super
          ar: empid
          permissionList: permissionId

where full_roads_payload is collected in a previous step

The check expression can be json path expression or can also represent a collected value from a previous or current step

shallowEqual

Shallow Equal is a way to compare two collected jsons to match and validate such that each element of A is present in the B (A is a subset B is a superset).

Property required type Purpose
sub required string the name of the object collected in this or previous steps that should be the subset
super required string the name of the object collected in this or previous steps that should be the superset
[key] optional / multi string key is the arrays in the object and value is the attribute of the object that should be used to compare and identify the object

example JSON A (subset)

{
    organization:"abc",
    address:"outer ring road",
    city:"bangalore",
    employees:[{
        emp: 123,
        name: "Ned Stark"
    },{
        emp: 234,
        name: "John Snow"
    }]
}

JSON B (superset)

{
    organization:"abc",
    address:"outer ring road",
    city:"bangalore",
    pin: 560102
    employees:[{
        emp: 123,
        name: "Ned Stark",
        cool: true
    },{
        emp: 234,
        name: "John Snow"
    },{
        emp: 345,
        name: "Sansa"
    }]
}

if test.yml

  shallowEqual:
    - sub: A
      super: B
      employees: empid

Result: Pass

Save

Save is a utility that allows the resoponse recieved from the step into a file. you have control to save the full payload or partial payload using jsonpath

example:

    save:
      $: "file_mongo_recs"
      $..name: "names_in_response"

More on Schema Validation

Define the location of the swagger file in your config/index.js with the name

 swagger: /config/swagger.yaml

In your check attribute you can add the name of the Model corresponding to the expected response.

 check:
   schema: Domain

The framework will add an assertion step to check if the schema of the response is matching with the schema as defined in the swagger. the assertion fails if the schema is not found or schema mismatch. the details of which all attribute are not matching schema is described in the test log JSON and viewable in the teakozi-viewer.

Running a subset of your test suits based on tags

in your index.js file

var tags_list = "a,b,c,d"
require("teakozi").start("project/github",working dir,{tag:tags_list})

run the test cases

$ node index.js

Chaining test execution

The start function returns a promise that is resolved when tests complete. you can chain further tests like following or handle the test end event for any other handling

var dir = "/home/jitendra/code/teakozi-examples"
var t = require("teakozi")
t.start("project/github",dir,{tag:"cars"})
  .then(()=>{
    t.start("project/github",dir,{tag:"swagger"})
  })

The test logs are shown on screen and the logs are written into the project directory/log with a timestamped folder containing all and individual test run information

Teakozi-Viewer

This supporting project enables you to view the HTML reports from the collected logs in the log directory (all.js). Please clone and install from the following github repository https://github.com/jpthethi/teakozi-viewer

Teakozi-Editor

Teakozi editor https://github.com/jpthethi/teakozi-editor is a companion to Teakozi framework. The editor helps in providing a GUI for creating and editing the yml files that describe the tests.

Feedback

Please submit your suggestions / bugs via Github. You are welcome to fork and submit back your pull requests

License

ISC

teakozi's People

Contributors

jpthethi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.