networknt / light-4j Goto Github PK

Need to upgrade to the latest Undertow version which supports response body dump.

This gives the caller more control to handle exceptions.

As most service to service calls with be secured by client credentials jwt token with 10 to 15 minutes expiration and signature verification is very expensive. In order to save some CPU cycles, it would be a good idea to put the token into a hashmap with value of expire time. The question is how to remove the expired token from the cache. Need to think though it.

This is a preparation to start light-java-rpc and light-java-graphql

Is there service discovery feature just like in spring boot?

We have a mask module to mask sensitive data from header, query parameter, path parameter and body. It works with JSON object with JSON Path with configuration. With the introduction of Body module that parses body of application/json to list or map, we need to update the mask module to work with list and map. Regarding to traverse the list or map, here is an example.
https://github.com/networknt/light-java/blob/master/sanitizer/src/main/java/com/networknt/sanitizer/SanitizerHandler.java

Light-Java is designed to work with swagger specification to generate the code and trigger scope verification in security and validation in validator. However, this make injection of common services with is not defined in the service swagger specification hard to be injected and by pass all the swagger validation. Need to find a way to inject server info which is the endpoint that output all information about components wired in and their configuration.

I read that GraphQL is a mid-term goal on the road map. I am interested in this.

From the docs I see that it seems to strongly encourage starting a project from a Swagger spec so I'm interested to understand how GraphQL support might be incorporated into this project (just at a high level)?

21:06:04.611 [XNIO-1 I/O-11] ERROR c.n.exception.ExceptionHandler - Exception:
java.lang.IllegalStateException: UT000139: Exchange already complete
at io.undertow.server.HttpServerExchange.addExchangeCompleteListener(HttpServerExchange.java:920)
at com.networknt.audit.SimpleAuditHandler.handleRequest(SimpleAuditHandler.java:100)
at com.networknt.body.BodyHandler.handleRequest(BodyHandler.java:106)
at com.networknt.security.JwtVerifyHandler.handleRequest(JwtVerifyHandler.java:174)
at com.networknt.swagger.SwaggerHandler.handleRequest(SwaggerHandler.java:88)
at com.networknt.metrics.MetricsHandler.handleRequest(MetricsHandler.java:133)
at com.networknt.exception.ExceptionHandler.handleRequest(ExceptionHandler.java:53)
at io.undertow.server.handlers.SetHeaderHandler.handleRequest(SetHeaderHandler.java:90)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:207)
at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:243)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:134)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:58)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:88)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:559)

config files will be served by config server with git as repo and secrets will have a template only and filled during deployment.

It takes a few seconds for failed service to be removed from client discovery cache. The shut down sequence should be:

1. un-register from consul registry
2. still handling requests for 30 seconds
3. shut down

This is necessary to support multiple public key certificates on light-oauth2. Also, for the resource service to verify a in coming JWT token, it won't need to iterate every key available but just pick the right key to verify the token. If token verification module cannot find the right key locally, it will call the light-oauth2 key service to get the public key certificate by kid.

Add support for maps of objects, each employing a set of properties
Revisit support for bean references

For OAuth2 Authorization Server, if client application render the SPA to collect user credentials, we need to make sure our Authorization Code service can handle CORS gracefully.

I have this cause (with version 1.2.7):
i've generated project by swagger, changed port in src/main/resources/config/server.json but port not change! Then i thought for change port need use param httpPort but it not help too.
Actually light-java framework has own server.yml (*.yml config is more priority) and my app not read my server.json

It's look like bug, is not?

yaml is easy to work with and can have comments. Config module will load file with yaml or yml first and then json. If nothing is found, then lookup the next config level.

Currently, we control cors at host level without checking methods. Need to add another level of control at method level when host is allowed. The cors.json need to be changed.

With the new service registry, discovery, cluster and load balance, the client module need to be updated as route doesn't make any sense anymore as the http calls will be based on the real ip instead of domain and connection pool management becomes more complicated on per IP basis. When you are thinking about local first load balance, things are getting more complicated. To make it simpler, we are going to try HTTP/2 instead of HTTP/1.1 for service to service calls. These days, most browsers support HTTP/2 so for the APIs that serve SPA, we should be OK.

To make the above refactor:

1. Enable server for HTTP/2
2. Update client to use OKHTTP instead of apache httpclient
3. Update client to server communication

For beans with properties, currently create the bean with default constructor and then set properties with the map passed in from service.json using Java reflection. It works for most of the cases, but doesn't work if you want to construct other object with the parameters passed into the constructor. For example.

    private ZkClient zkClient;

    private String zkServers;
    private int sessionTimeout;
    private int connectionTimeout;

    public ZooKeeperClientImpl() {

    }

    public ZooKeeperClientImpl(String zkServers, int sessionTimeout, int connectionTimeout) {
        zkClient = new ZkClient(zkServers, sessionTimeout, connectionTimeout);
    }

Need to gracefully down grade the behaviour to something reasonable but not give errors. Or this is just an education issue? It is a good idea to enforce security if people want traceability and metrics?

As service is discovered and IP is not known during configuration, should we just remove per route configuration or leave it to support static lookup services?

Need to gracefully down grade the behaviour to something reasonable but not give errors. Or this is just an education issue? It is a good idea to enforce security if people want traceability and metrics?

When a service start the first time, it goes to central config server to load configurations and then cached them locally. All subsequent startups will get config from config server but if the server is not available, use the local copy to start. The local copy is just a backup in case a black out recovery and thousands services start at the same time and over load the config server or config server is not up yet.

Need to check the null and throw an ApiException so that it will return to the consumer.

The (default) com.networknt.config module implements a file-based config-properties system.
Any idea how hard it would be to re-implement using S3 or some other mechanism (like HTTP
to an external server)?

With configuration, limit number of requests within a short period time from the same IP address. Although we have very high throughput but we do want to fail fast if there are any potential attacks.

As Undertow is using jboss logging and we are using slf4j with logback in our framework, we cannot see any log from Undertow even errors. In order to make Undertow use our slf4j and logback config, we need to add a system property in the command line or just add a line in Server.java.

I think the second option is much better as we always need it.

        System.setProperty("org.jboss.logging.provider", "slf4j");

In order to capture all the exceptions in ExceptionHandler, dispatch in the exception handler as suggested by Bill O'Neil.

I'm a little concerned with http://localhost:8080/… being hard coded in the generated tests. I did have an issue where the tests can't run if the development server is running locally.

It seems like the TestServer class could assign a random port… and the tests could query TestServer.getInstance().getBaseUrl() or something.

At the least, the test server should run on a different port. (Maybe a src/test/resources/config… file will do this). I'm going to try that approach.

I have read the doc hosted in https://networknt.github.io/light-java/. Light java will be a great framework to help developers create microservices with good architecture. I have being using Jhispter for a while. Although it's based on spring, which is too heavy for microsevices, the speed to create a project from scratch is still astonished me a lot.

One of my clients has to deployed service on data centre due to private cloud availability. Without Kubernetes to manage the secrets, we need to find a way to encrypt the secrets(password, client_secret, private key etc) in config files the first time they are loaded.

This gives us more opportunities to attach more micorserivces to the stream.

In my code:
HttpServerExchange.getRequestReceiver().receiveFullBytes((exchange, data) -> { //do stuff with the data DATA IS EMPTY }, (exchange, exception) -> { //optional error handler } )
How can I get the body or the serialized JSON Object passed in my POST request? data is empty.
Thanks,
matt

this doc says:

When you start a service, you might realized that every few minutes, “InfluxDbReporter report is called” will be shown up on the console. By default, all services will try to report metrics info to Influxdb and subsequently viewed from Grafana dashboard.

Is there a way to get InfluxDbReporter report is called to not show up on the console?

With service to service call, if token is not matched, the error message is not clear when service has the wrong token and what is the token scope and spec scope. Need to output service id and scopes to assist developers for debugging.

error response.

{"statusCode":403,"code":"ERR10005","message":"AUTH_TOKEN_SCOPE_MISMATCH","description":"Scopes in authorization token and spec scopes are not matched"}

The Sanitizer is too aggressive and mangles content strings. I understand the intent, but had to disable it to retrieve the original strings sent in. In my case I am passing an XML document as a string.

Before disabling sanitizer: <?xml version=\"1.0\"?>
After disabling sanitizer: <?xml version="1.0"?>

In generated src/main/resources/META-INF/services/com.networknt.handler.MiddlewareHandler

#com.networknt.sanitizer.SanitizerHandler

I was looking at the web framework benchmark (https://www.techempower.com/benchmarks/previews/round14). light-java has really good performances in Fortune & Single Query. But in Multiple Queries & Data Updates light-java's performances are less than exemplary, falling behind even some Ruby & Python frameworks. Any particular reasons why that is so?