Overview:

Generate swagger docs from the API actions/routes.
All of the type information and routing structure is available in the ApiReg and ApiRegAction data structures

Overview

Universal APIs support middleware in the form of the items below.
This enhancement cleans up the middleware components by making them interfaces and makes them reusable for either a single API or globally for a Multi-API service.

1. Hook : Hooks for on before and on after processing of an api call
2. Filter : Filters in/out an api call
3. Error : Handles an error during an api call
4. Rewriter : Rewrites the request path/parameters ( e.g. to support REST / etc )

References

All the middleware is located in https://github.com/code-helix/slatekit/tree/master/src/lib/kotlin/slatekit-apis/src/main/kotlin/slatekit/apis/middleware

Overview

Currently there are smart strings ( Email, PhoneUS ) in the Slatekit common project.
These need to be documented as part of the utilities page.

References

https://github.com/code-helix/slatekit/wiki/Docs

Overview

Currently, any item returned from an API action is converted to an HTTP response by the Spark Server. Some of the methods/actions return a Result which models successes/failures through the project and is used as an alternative to Either/Option etc. The Result is very useful, however, a proper Response type is needed in order to be able to setup metadata ( e.g. headers ) if the API wants to.

Changes

• Create a Response class that is similar to Result but has a meta data field
• Support converting from this Response to a Http Response

Overview

Currently, you can use smart strings as parameters for Universal APIs in Slate Kit.
You can setup a parameter on a action/method to be say PhoneUS ( an smart/enriched string ) with a predefined format. The support for parsing request parameters as normal strings to smart strings is in the APIs component but has not been tested. This is tested in the unit-tests but only for CLI mode. This ticket is to ensure smart strings work for the web as well

Changes

1. check smart string support for web
2. add/edit unit-tests as needed

Links

http://www.slatekit.com/kotlin-mod-smartstrings.html

Overview

Create a new light-weight Gate component that is similar to a circuit breaker in the SlateKit.Core project.

Features

• track request total
• track request count in a batch
• track error total
• track error count in batch
• track error ( last one )
• open on start
• close on error percentage too high
• close on request percentage too high [WIP]
• reopen using exponential time values
• reopen with a specified time in seconds
• alert via callback
• attempt method that tries to call a supplied function

Overview

The redesign of the workers component is not complete. This ticket captures the remaining work.

Details

Add support for the following:

1. support new terminology/lexicon ( e.g. jobs/tasks/queues/workers )
2. support co-routines/suspendable operations
3. support use of channels for communication and thread-safe operations
4. support start, stop, pause, resume operations safely ( via channel communication ? )
5. support self-managed workers ( e.g. workers without a job queue )
6. improve support for stats/diagnostics

Overview

This fix involves optionally supplying a naming convention ( via the Namer interface in slatekit.common ), which will allow customizing the names of database tables and columns.

Use-Cases

• Supply an optional Namer to Entities, EntityMapper, EntityMapper and Model field builder
• Change the design of the Namer class to support simple naming convention
• Support for Upper/Lower Camel Case, Hyphen, Underscore cases

Overview

The AWS Queue discard/complete function is broken due to recent changes to allow type safe queue entries.

Details

In the discard method, the following cast breaks as item is now QueueEntry and to get the actual SQS message, you have to use item.raw.

  private fun discard(item: QueueEntry<T>, action: String) {
        when (item.raw) {
            is Message -> {
                execute(SOURCE, action, data = item, call = {
                    val message = item as Message
                    val msgHandle = message.receiptHandle

                    sqs.deleteMessage(DeleteMessageRequest(queueUrl, msgHandle))
                })
            }
            else -> {
                throw Exception("Incorrect QueueEntry for AWS Queue")
            }
        }
    }

Reproduce

1. Set up the AWS queue
2. Add an entry
3. Get an item and try to complete it

Expected

The entry should be removed from the queue

Actual

The code throws an exception and so the entry is always in the queue

Versions

all

System

Indicate the system that the issue occurs in

1. all

Overview

Currently, Universal APIs are loaded by checking for annotations on regular methods.
This enhancement involves support alternate registration methods.

Changes

• Allow registration of the actions from a trait
• Allow registration of the actions declaratively ( like http routes but with method references )
• Allow registration of the actions from a file ( json format that matches the fields in annotations )

Note

For item 2 in changes:

...

// Case 1: Register one at a time
actions.register( "admin", "users", "create" , method = Users::create )
actions.register( "admin", "users", "update", method = Users::update )
actions.register( "admin", "users", "delete" , method = Users::delete )

// Case 2: Register in bulk if actions match method names
actions.register( "admin", "users", listOf(
  Users::create,
  Users::update,
  Users::delete
))

// Case 3: Register using an api trait
// This will take all the methods in the trait and set them up as API actions
actions.register( "admin", "users", UsersApi::class )

// Case 4: Register using a file 
actions.register( "user://myapp/conf/routes.json" )
...

Overview

Currently, the APIs in slate kit support auto-converting request parameters ( querystring / json ) into parameter values for the methods on the classes. There is also support for getting the actual request and meta data as parameters as well. This ticket allows users of the APIs to supply custom converters that can handle the conversion of request data to method parameters.

Changes

• Add a new component called type converter to handle custom conversions
• Allow the type converter to be associated with a specific type
• Add a way to supply a list of converters during setup of the APIs
• Hook the type converters into the deserialization phase

Overview

Docs for components need to be updated before launch ( mostly the website )

Details

The major components need to have docs and or updated docs

Items

1. app
2. cli
3. entities
4. workers
5. apis
6. server

Overview:

Integrate the Universal API approach with the background workers.
This allows running a request ( a universal api request ), from a message queue.

1. An api request serializes the request method and parameters into json and puts the json into message queue
2. A background worker gets messages off the queue
3. A background worker dispatches the request back to the API system

Overview

Currently, the setup/deployment of Slate Kit applications use raw binaries ( e.g. JRE, Kotlin Libraries, etc ). This enhancement is to support both raw deployment and also a Docker based set up.

Dependencies

• JRE 1.8 and greater
• Kotlin 1.2.31

The API approach in Slate Kit is Protocol Independent, but rephrase this as Universal APIs in the site docs and respective locations

Overview

Currently, modeling of successes and failures in Slate Kit is implemented using Result that has 2 branches ( a Success and Failure branch ). While this is useful it is not ideal. This change makes the Result more generic by making the failure branch support any type. This this becomes Result<T,E>.

Notes

• The Slate Kit Result<T,E> also has support for a custom status code, message and success flag
• This mimics the Rust language Result<T, E>
• The is almost a hybrid of an Either/Try in Scala

Changes

• Support Result<T, E>
• Success biased
• Mimic an Either<L,R> and is Success biased
• Get rid of older "tag" field
• Get rid of the "get" method ( to avoid null potential ) and replace with fold/getOrElse
• Use inline for the map, flatMap, fold, methods
• Add type alias for ResultMsg = Result<T, String>
• Add type alias for ResultEx = Result<T, Exception>

The current docs generator is in Scala. Move this code to kotlin ( fairly small effort ) and keep this in a new slatekit-tools project.

Overview

Allow specifying a field as being:

1. a unique field
2. an indexed field

Changes

• Add attributes to the field annotation/class
• Add sql code generation

Notes

A separate fix is needed for customization around id field ( ticket will be created )

Help

1. Show comprehensive CLI / Api integration docs on startup
2. Support for Meta/Sys arguments

Sys options

Support various sys options which are essentially options available to any CLI command. These sys options represent flag/options supplied to the command itself. Examples are output format, loading inputs from a file, etc. The full list is :

 $file    = loads inputs from the file path
            ? optional "user://myapp/scripts/setup.txt"
 $sample  = generates a sample request file in the output directory
            ? optional "sample1.json"
 $format  = the format of the output
            ? optional [ json | csv | prop ]
 $log     = whether to log result to output directory
            ? optional true | false

Overview

There are a few different components to the Universal APIs ( listed below ).
These should all be made pluggable ( e.g. such that different implementations could be injected into the API system ). Most of this is already working to fair degree. They components have to more formally setup with traits, default implementations, setup parameters for the API.

Components

• Converter: Converts each parameter from a raw string/json type to a Kotlin datatype
• Serializer : Converts the return value of an API call to a JSON value
• Deserializer: Converts the source request parameters to method inputs

Notes

• The converter already exists as a class
• The serializer already exists in the slatekit.common and slatekit.meta projects
• The deserializer needs to be extracted out of other classes

Overview

The current implementation of Result uses an interface. Refactor this to be a sealed class to ensure restriction to Success / Failure branches.

Links

• https://kotlinlang.org/docs/reference/sealed-classes.html
• http://www.slatekit.com/kotlin-mod-results.html

Overview

The routes in the APIs ( setup as annotations ) are ultimately stored in a collection for easy lookup.
But there should be a Router/Routes class to make this storage / lookup even easier. And the Router/Routes system should be a first class component of the Universal APIs

Changes

• Create a routes class to store all the "routes" via "Area", "Api", "Action"

Notes:

• Refer to Areas

Overview

The current argument parser supports:

1. regular parameters via - prefix via -level=warn
2. meta parameters via @ prefix via @token='abc'

Changes

Add support for a 3rd type of parameter.
This 3rd type is considered a sys parameter and uses the prefix $ via $output=json
This allow for parameters to be exclusively used for the CLI component

Setup Travis/Circle CI automation

1. build using existing gradle setup ( pretty easy )
2. run tests using gradle test on project slatekit.tests
3. only have to remove some tests since they rely on the database

Overview

Ensure support for aliases via a double prefix e.g. --

Given

1. Sample argument name = env, alias = e
2. normal input would be -env=dev
3. alias input would be --e=dev

Changes

• Support for specifying an alias for argument
• If schema is provided, transform the aliases to the canonical normal, long-hand name

Overview

Enable support for a run mode when launching the cli so it can run in 2 modes:

1. cli : default mode with user interaction
2. script: runs a scripts supplied on the cli and exits

Example

SETUP
app-cli.jar 
	-auth="user://app/login.conf" 
	-mode="script"
	-script="user://app/create_users.json" 
	-log=true 
	-output=true
	-env="local:dev"

Overview

The Slate Kit Tools project is a Command line program for various tooling around Slate Kit. One of the main functions is to generate the Documentation for all the utilities and some of the architecture components. It actually runs on Universal APIs, so you can call api actions/commands to run some code. This enhancement is to add a new Project API to create new projects in Slate Kit using the sample apps as a reference.

Changes

• Add a project generator API
• Add support for creating an app project
• Setup templates for the respective projects
• Have the API generate a new project using one of the templates

Options

These are possible arguments for the API to create a new project

Overview

Create a new Alerts component that can send out generalized alerts to various targets/destinations such as a Slack Web Hook | PagerDuty | OpsGenie.

Details

An Alerts component that can send out generalized a generalized Alert or Notification to various destinations. This is allow for notifications/alerts of important events in an application. One of the easiest integrations is to send an alert to a Slack Web Hook. Other potential integrations are sending alerts/incidents to PagerDuty or OpsGenie. A simple alert can have the following structure:

alert : {
      area: 'app.registration.new',
      name: 'NEW_DEVICE_REGISTRATION',
      uuid: 'abc-123-xyz',
      desc: 'User registration via mobile',
      success: true,
      code: 200,
      tag : "a1b2c3",
      fields: [
          { title: 'region' , value:'usa'       , pii: false },
          { title: 'device' , value:'android', pii: false }
      ]
  }

Area

Diagnostics and monitoring

Problem

Currently, in Slate Kit ( and our applications using Slate Kit ), there is no way to easily send out structured events/alerts to external systems to notify recipients of important activity occurring in an application. This is needed as a diagnostics / monitoring feature.

Solution

A simple alerts component that for the time being can integrate with PagerDuty and/or Slack Web Hooks for a start.

Alternatives

Potential integration with OpsGenie and PagerDuty are idea. However, there should be a distinction between severity level of the alerts so as to not trigger Incidents.
Aslo, note that structured logs can serve the purpose to some degree as they can be set up to go to loggly and alerts in loggly can be set up to check for specific log messages. However, there should still be some direct alerting component available in Slate Kit and be used at the source of an important event in an application.

• support field type java.util.UUID
• support field type slate.common.UniqueID
• support registration options ( type only, type / repo , type / service )
• support getting relation ( 1 )
• support getting relations ( many )
• support nested objects

Overvew

Files can currently be downloaded, this is just to ensure that the upload of files / images and download of files/images works fine in the Http APIs.

Overview

Currently, there are a few different middleware components ( Filters, Hooks, Errors, Rewrites )
This ticket is to add support for a new middleware called "Handler" which if implemented handles the processing of the event instead of the API Container ( which would otherwise normally flow to the execute method to call the API method associated with the request.

Goals

The goal of this feature is to support customized processing of the request. E.g. this could be many different things. The current use case is to transform the request in the some way ( currently needed for mobile server project ).

Notes

The middleware are interfaces that when implemented, allow global handling of requests is specific ways. E.g. Filter middleware filters the request, Hooks run before/after requests, Errors handle errors and Rewrites rewrite the route. This new "Handler" method would follow the same principle.

Changes

• add a new tag field on the Action annotation ( to allow clients of Handler to filter on tag )
• add a new tag field on the Action data class to store the same tag from annotation
• add a new Handler middleware component
• support serialization of request to JSON ( metadata/queryparams/post data )
• support new simpler method to send a request ( now in JSON form ) into a queue

This works.

1. Checked download from maven of slatekit.common in small hello world project
2. Setup page on www.slatekit.com is now updated. http://www.slatekit.com/kotlin-setup.html
3. Setup page now emphasizes gradle as primary setup option
4. Note, the maven url must be setup.

repositories {
    mavenCentral()
    maven {
        url  "http://dl.bintray.com/codehelixinc/slatekit"
    }
}

dependencies {
    compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
    compile "org.jetbrains.kotlin:kotlin-stdlib-jre8:$kotlin_version"
	
    // Reference the slate kit binaries here
    compile 'com.slatekit:slatekit-common:0.9.9'
    compile 'com.slatekit:slatekit-meta:0.9.9'
}

Overview

Currently, the API component supports code generation of client side API proxies in Java.
This is to enhance the code generation for Swift ( for IOS )

Changes

• check code generation ( its been a while )
• write a code generator for Swift
• write a code generator for Java

Overview

Support for handling Enum in multiple components

Changes

Support for enum in the following areas

• Reflection: Dynamic parsing of enum types
• ORM : For Model fields
• Serialization: To/From Json
• API: For parameter conversion

The App Context currently holds info for:

1. host info
2. language info
3. about

Add another type called Build that has build info available for runtime.
This includes:

1. commit id
2. branch info
3. version number
4. date of build

Make this info optionally available and load it into the context

Overview

Currently, the Universal APIs support accessing the Request as a parameter. This enhancement allows also accessing just the meta data of the request. This also fixes a few bugs associated with these 2 parameters ( Request, Meta ) since they are auto-supplied by the framework.

Notes

Fix bugs to address:

1. Code generation to exclude Request/Meta arguments in the action methods
2. Discovery of the arguments
3. Validation of the arguments to exclude request/meta

    // Get access to the Request.meta ( metadata ) property
    @ApiAction(desc = "", roles = "@parent", verb = "@parent", protocol = "@parent")
    fun hello(greeting: String, meta:Meta): String {
        return "$greeting back"
    }

Overview

Miscellaneous cleanup, removal of warnings refactoring, monadic calls using Result where applicable.

Changes

• ensure use of constants
• remove all warnings
• use result's map/flatmap
• rename fields are needed

Hi guys, thanks for all the good work being done here. I am waiting anxiously for 1.0 release :)
I have a question regarding auth in slatekit. In the docs it is written that you leave handling auth to other libraries. While this is good idea, maybe you have some suggestions on what would be be the best library choice for quickly implementing it both on server and client side alike.

For example, in case of Firebase the selling point for me was drop in auth solution including social auth and client side libraries they prepared. With this I am able to include auth in my app in around 5 minutes total.

Slatekit could really benefit from having some ready solutions or samples regarding auth

Overview

Running the slatekit.app giving it arguments using an empty line or string results in a failure of the app. This was working before.

Details

When using the App component and supplying command line arguments that are empty or not supplied, the App fails as the parsing of the arguments results in a Failure.

Reproduce

1. Use App
2. Use empty command line args
3. Run app and it should fail

Expected

If the args are all optional, it should be able to handle empty line, or 0 args.

Actual

App component does not run and fails

Versions

All

System

Indicate the system that the issue occurs in

1. all

Misc cleanup

• config loading
• error handling
• field names ( breaking )

Overview

There is an abstraction for messaging/push notifications in the slatekit.core project.
The location is https://github.com/code-helix/slatekit/tree/master/src/lib/kotlin/slatekit-core/src/main/kotlin/slatekit/core/push

Changes:

Change the message structure from a flat set of fields to a more structured format such as having sub-objects for sender, recipient, content etc.

Overview

When using the slatekit.results component for modeling success and failures, the method to convert the Result<T,E> to a Http compatible status code fails as it doesn't handle all the built-in status codes.

Details

The conversion is specifically missing all of the success codes such as Confirm

Reproduce

1. Build a Result<String, String> with status code confirm
2. Run the StatusCodes.toHttp method

Expected

The http status code should come back with the appropriate http code for confirm

Actual

You get back a 1010 status code

Versions

all

System

Indicate the system that the issue occurs in

1. all

Additional

There should also be a method to convert from http back to the status built-in status codes
Essentially a reverse approach

Overview

Improve the logging support by using an external provider but having a simple console logger as a default. Ensure that slatekit.common still has 0 dependencies

Changes

• add a new slatekit.providers project with support for Logback adaptors and dependencies
• add a new Logs component that can get a logger
• add examples of logback configuration in the sample projects

Overview

When installing all the modules, there is a seed method which will seed data for all modules, but the seedModule method is private.

Details

This is needed to be able to only seed data for a single module or to cherry pick re-seeding of data for 1 or more modules.

Reproduce

1. step 1
2. step 2
3. step 3

Expected

Expose the seedModule method

Actual

Its private

Versions

all

System

Indicate the system that the issue occurs in

1. all

Overview

The Universal APIs in Slate Kit are organized into 3 hierarchical parts in the format area.api.action such as app/reg/registerNewUser.
There are several built-in APIs provided by Slate Kit which are organized into various areas but the area names are somewhat confusing and also the apis may be better suited in other top level areas. This ticket does the following:

1. Rename top-level areas
2. Reorganize some of the built-in apis into the new top-level areas.

Changes

• area: app for application specific info/metadata ( e.g. the AppApi
• area cloud for all apis revolved around files/queues/email/sms ( Note, email/sms are using the SendGrid and Twilio API )
• area infra to host many of the existing APIs
• area slate for slatekit specific API/Tools

INFRA:

infra/cache
infra/cmd
infra/config
infra/encrypt
infra/entities
infra/errors
infra/logs
infra/modules
infra/workers
infra/host

APP

app/about

CLOUD

cloud/email
cloud/files
cloud/push
cloud/queues
cloud/sms