• build
• test
• NTH: validate coverage over 80%

After #40 will be completed we need to use
Script content saved in CRD in API server to run new execution

executor api on posting new script execution
will return 201 - Created and store script execution info in storage.

in parallel workers will check in some intervals what scripts need to be run.

Atomic action from storage need to be supperted (in Mongo we have FindAndModify which lock document)

Starts new script execution

use some colors for kubectl

(zap looks like good lib to go with) - used by exu previously and by kubebuilder - os we will have only one logger dep.

as:

• some net.http endpoints with preconfigured routes
• or maybe some popular frameworks middlewares
• ?? ... or other ideas :)

Get all script executions

Postman executor:

• run newman process
• get output as string
• implement start and
• get status enpoint (after storage implementation)
• run should create ID for execution (in separate task - imlpement storage)
• NTH: limit parallel runs in node to some constant (4?) - separate task

Define project structure inside kubetest repo, use golang standard repo as skeleton

TODO:

• code structure (look at golang standard)
• health endpoint
• Dockerfile
• as separate go module inside kubetest repo
• NTH consider some simple client code for use inside operator
• NTH: consider GRPC (is it needed for MVP?)

Curl based simple one-node executor, status storage in-memory - for documentation purpose

1. create executor from template
2. clear mongodb
3. implement runner (create CURLRunnel based on Example runner)
4. remove worker (we will go with single node)
5. run inside Start endpoint as goroutine

(with plantUML inside project)

make example draft for high level architecture

closes #60

• deployment
• service
• Mongo db

consider use helm or other templating stuff

see #31 first

We need to implement common executor result format

• result owned by kubetest
• could be quite complicated as need to handle all possible use cases
• executor will need to implement it
• renderer will use common output format to return result in given format (text, junit, json etc)

see #66

Kind: KubeTest
Type: PostmanCollection
Content: inline json content

Descriptions how to write / (probably later - generate) custom executor

• based on opeanapi executor definition
• as simple as possible (w/o scaling issues internal queues storage etc, run on single node)

Several problems need to be addressed:

• How to allow external developers to add new executor

It would be great to have common error structure for all rest endpoints

Currently most errors are not handled at all

There is several libraries for errors based on
https://github.com/moogar0880/problems
https://github.com/mschneider82/problem
https://pkg.go.dev/github.com/longkai/rfc7807

moogar0880 looks like easiest one to use

Introduced in #23 need to be used everywhere

• TODO create some easy to use wrapper over http response function (like gofiber c.JSON() function)

for POST /executions/ on API Server and for executor

I'd prefer a design-first approach to our API interfaces - both internal and external

Add openAPI specification in api folder.
For now endpoints

• GET /tests gets all tests available (simple w/o paging and filtering
• POST /tests creates new CR based on CRD
• PUT /tests/:id updates given test id with test data

Q: are tests good name ?

KubeTest needs some licence before going public.

Define actions for kubectl kubetest plugin

1. Starting new script execution

$ kubectl kubetest scripts start SOME_SCRIPT_ID_DEFINED_IN_CR

Script "SCRIPTNAME" started
Execution ID 02wi02-29329-2392930-93939

• watch for results immediately ?
• show some output from run ?
• maybe allow to name/describe your execution (will be easier to check) for example we can run execution for different server config when debugging some issue so we would have several executions (testing_128M testing_200M testing_256M testing_512M)?

2. Aborting already started script execution

$ kubectl kubetest scripts abort SOME_EXECUTION_ID

Script "SCRIPTNAME" Execution aborted

3. Getting available scripts

$ kubectl kubetest scripts  list


ID         NAME              Type
040-134   HomePage test      postman-collection   
123-246   Contact API test   postman-collection

4. Getting available executions

kubectl kubetest scripts executions

ID         NAME             Status     Complete   Start              End
1233-333   HomePage run     pending    75%        2021-07-30 12:33   
1233-332   HomePage run     pending    100%       2021-07-30 12:33   2021-07-30 13:10

What output renderers ?

• plain text (for the beggining) but prepare model for adding additional renderers
• json
• go template

Get script execution

2 things to fix

• move main entrypoint to cmd/operator
• find way to fix module in module (probably we'll need to remove mod inside or split project into separate modules side-by-side

Use it as app inside kubetest repo

• generate operator
• use API Server enpoint to pass info about changes in CRD resources.

Add flag for output format,
design common output functions
for all kubectl kubetest plugin commands

like kubectl -o

• text
• json
• go (text-template)

blocked by #54

Put run scripts for mockserver (or other solution?)

DELETE /scripts​/{id}​/executions​/{executionID}​
Aborts execution

we can implement this later and provide instructions on how to do this "manually" for now

TODO - update docs + API spec

Do we really need this?

As we want to be future proof - e.g. probably we could split modules into separate services it would be great that each binary would be loosely coupled with core.

Check if all requests/responses/objects are in sync with code:

• object schemas
• request bodies
• API responses

As tester/engineer - I want to simply import my json Postman collections file to kubernetes cluster - next run test

TODO:

• implement POST /scripts - create new Script CR
• implement API Client for scripts to create new CR
• implement scripts import kubectl command (allow importing from stdin

usage:

kubectl kubetest scripts import --file=my_collections.json
#or 
cat my_collections.json | kubectl kubetest scripts import - 

Getting all scripts available in CR

simple method for use inside API for getting CRD details

So to draft what should you do:

• create /pkg/client/script.go
• create simple struct with methods ListScripts() []spec.Script (adjust types for real ones 🙂
  )
• create method GetScript(name string) spec.Spript
• implement all methods to work with cluster
• as additional NTH training - check how to test those lib (maybe client-api have directions how to test those endpoints).
• check how to configure it (on your example config is in cluster config - we need to decide how to use it also on dev machine - it’ll be not in cluster

as discussed - a script showing kubectl commands required to demo the MVP

For start:
executions count (group by script name)

• total, failed, success
• per script or for all scripts