netflix / karyon Goto Github PK
View Code? Open in Web Editor NEWThe nucleus or the base container for Applications and Services built using the NetflixOSS ecosystem
License: Apache License 2.0
The nucleus or the base container for Applications and Services built using the NetflixOSS ecosystem
License: Apache License 2.0
The dependencies such as 'javax.ws.rs:jsr311-api:1.1.1' and 'javax.servlet:servlet-api:2.5' are not required by all sub-projects but are defined globally or in core module.
We should move this to the modules which need them.
Embedded Jetty may not always be the ideal way to host the admin ui. Consider dependencies on the jetty servlet api, colliding ports, etc.
Feature request: Make Jetty optional, and provide a second option for hosting via juice servlet instead. Would you accept a pull request for something like this?
Thanks,
Rob
Currently, Eureka health is determined here: https://github.com/Netflix/karyon/blob/master/karyon-eureka/src/main/java/com/netflix/karyon/server/eureka/EurekaHealthCheckCallback.java#L55 which is not extensible.
Ideally one should be able to override just this assertion according to their own logic.
Hello,
Having an issue with custom appender that may (?) be related to karyon/netflix oss...
I have a custom appender configured like this:
log4j.debug=true log4j.rootCategory=DEBUG, stdout, audit log4j.appender.audit=com.liaison.framework.audit.AuditAppender log4j.appender.stdout=org.apache.log4j.ConsoleAppender log4j.appender.stdout.layout=org.apache.log4j.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%m%n
It works (gets called for all root category log calls) at the beginning of bootstrapping during server startup (jetty via jettyRun) until these statements appear...
04 [Daemon] INFO com.netflix.config.util.ConfigurationUtils - Loaded properties file file:/Users/rob/git/service-seed/service-implementation/build/resources/main/hello-world-dev.properties 115 [Daemon] INFO com.netflix.karyon.server.ServerBootstrap - Creating a new governator classpath scanner with base packages: [com.netflix, com.liaison] 665 [Daemon] INFO org.hibernate.validator.util.Version - Hibernate Validator 4.1.0.Final 692 [Daemon] INFO com.netflix.karyon.server.HealthCheckModule - No health check invocation strategy specified, using the default strategy com.netflix.karyon.server.eureka.AsyncHe
After this point, the binding between RootCategory and AuditAppender is lost.
Question is, is there anything fancy going on with respect to logging within the NetFlix projects?
Thank you,
Robert Christian
With archaius properties file config of:
com.sun.jersey.config.property.packages=com.acmeair.services.authService
com.sun.jersey.api.json.POJOMappingFeature=true
PropertyBasedResouceConfig initIfRequired exhibit this behavior:
Map<String, Object> jerseyProperties = createPropertiesMap()
always comes back an empty map.
It also looks like debugging the code that the code called here:
Properties properties = getConfigInstance().getProperties("com.sun.jersey");
Is wrong. Are you sure the underlying code isn't for getting a multi-value "com.sun.jersey" property vs. getting any property that starts with com.sun.jersey?
NettyToJerseyBridge
passes an empty buffer for the payload to jersey.
It should pass a buffer which gets populated from the content in HttpServerRequest
If a custom health check invocation strategy is provided via the property: "com.netflix.karyon.health.check.strategy", karyon does not bind the appropriate class.
On adding in KaryonEurekaModule to my Modules, I get:
It looks like Amazon is assumed by
ApplicationInfoManager initialization to EurekaInstanceConfig -> implemented by CloudInstanceConfig. This seems like this is the Amazon one. I tried to do the older Karyon 1.0 approach of com.netflix.karyon.eureka.datacenter.type=MyOwn in my config properties and as a -D parameter, but neither avoided the issue.
right now, it only supports UP/DOWN status.
Setup:
Trying to use karyon 2.0.02 in a Java module using maven build system.
Issue:
Build always fails trying to look for eureka-client artifacts of version "1.+".
My analysis:
Looking into the pom for karyon-eureka version 2.0.02, some versions are specified as "1.+" and "0.+" which to my knowledge these are not valid wild cards in maven. I could figure out that these are dynamic versions in gradle but it is not valid in a maven pom file.
karyon-users conversation:
https://groups.google.com/d/msg/karyon-users/A37Wd0stGJg/RCCzchplTRcJ
Feedback from Google groups:
Yes what you encountered is an issue w.r.t. using karyon with maven. I will try to
figure out a middle ground in which wild card dependency versions can be specified
in karyon interoperably across maven,ivy, gradle. In the meanwhile can you file a git
issue for this?
mostly-likely implemented inside a servlet filter (ie. NFFilter internal Netflix name)
Th example currently uses the stable release artifact. It is good to validate the release but it adds a point of confusion & problems in testing.
We should depend on the latest changes in the project.
While developing my webapp using Karyon, I've noticed that component classes must also be marked as @Singleton
to avoid being instantiated for each injection.
Is this intended behavior?
Thanks!
Consider baking WebSocket support into Karyon, with a HelloWorld example.
the jyst of the issue is that sensitive system properties are exposed . this is definitely a user error that can be locked down through AWS security groups and such, but i think it's significant enough to address with a workaround - perhaps a filtering mechanism to allow a developer to specify certain properties that are not accessible from this admin interface - or any REST interfaces into this admin service.
i noticed this when i went to open up port 8077 for the /healthcheck functionality. unfortunately, once i open that port, my AWS credentials are exposed because i pass them in as System props.
the other option is to allow a developer to lock down particular endpoints of the server listening at port 8077. in this case, i would allow /heathcheck, but not /adminres, for example.
i've documented this on the wiki at the following location: https://github.com/Netflix/karyon/wiki/Karyon-admin#access-control
Currently the code is in karyon-governator and hence forces people to have compile time dependency on eureka specifically even if they are not using it.
Contains only manifest and not KaryonGuiceContextListener.
This causes runtime failure at Jetty startup.
http://mvnrepository.com/artifact/com.netflix.karyon/karyon-extensions/2.0.02
To support running Karyon based applications in environments like Docker containers it would be useful to package the application as either an executable jar, or as an archive containing a startup script and all the jars on which the application depends.
This means we would need an embedded container inside the application rather than deploying the app inside a container. As long as there is a main class which bootstraps the application, there are various ways of achieving the packaging -
a. Gradle's application plugin
b. Maven's shaded jar
c. Leiningen's uber-jar
d. One-Jar
Does this maybe have to do with the transition from 1.x to 2.0? It seems like some things from 2.0 have been merged but this may have gotten missed.
warning: [options] bootstrap class path not set in conjunction with -source 1.6
/Users/gorzell/dev/git/oss/gorzell-karyon/karyon-core/src/main/java/com/netflix/karyon/server/ServerBootstrap.java:34: error: package com.netflix.karyon.server.eureka does not exist
import com.netflix.karyon.server.eureka.EurekaHandler;
^
/Users/gorzell/dev/git/oss/gorzell-karyon/karyon-core/src/main/java/com/netflix/karyon/server/ServerBootstrap.java:153: error: cannot find symbol
return EurekaHandler.class;
^
symbol: class EurekaHandler
location: class ServerBootstrap
2 errors
1 warning
:karyon-core:compileJava FAILED
Currently, the guice module HealthCheckModule is required to be used to bind an implementation of HealthCheckInvocationStrategy and HealthCheckHandler.
It will be nice to provide @ImplementedBy in both these interfaces pointing to default implementations.
The interfaces are identical with the exception of the method name. Moving things to one interface should simplify things and make interoperability with RxNetty easier.
Currently, karyon is tightly coupled with eureka and there is no way to define a custom service registry.
It will help if the ServiceRegistry is abstracted and plugged in.
The basic idea will be a way to register/unregister with the registry, rest of the operations , if any, can be directly invoked by the application.
The pull request #45 introduced a regression that stopped calling beforeInjectorCreation() on the ServerBootstrap.
The helloworld example and any jersey based app depends on this particular method be called.
The README link to hello-netflix-oss is broken here: https://github.com/Netflix/karyon/tree/master/karyon-examples
The "Running the example" section here: https://github.com/Netflix/karyon/tree/master/karyon-examples/src/main/java/com/netflix/karyon/examples/hellonoss/server isn't correct anymore w.r.t the location of this README.
The build file should possibly take the name of the example to run.
The health check should be available on the same port/listener that is handling the service requests. Otherwise you have a condition where firewall rules or the listener dying could cause the service to fail while the healthcheck still passed.
From the jersey-bundle description:
Such a bundle is only intended for developers that do not use Maven's dependency system.
Consider specifying exactly which Jersey dependencies are required.
Please ๐
If the user of Karyon has more control over KaryonServer initialization, it is a round about way to specify the ServerBootstrap instance as a property.
This results in being forced to pull in eureka dependencies even when you disable the usage of eureka, and should probably be refactored so that this isn't the case.
Karyon currently requires the ServerBootstrap class to be specified via a property. It then internally allocates this via Class.newInstance(). Seeing as Karyon is already Guice-based it would be much better to have Guice inject the ServerBootstrap. This way, Karyon users can have injections into their ServerBootstrap instance.
AdminResourceContainer does the jetty server initialization in the constructor.
Governator does not initialize the fields annotated with @configuration till @PostConstruct.
So, no fields configured by governator are properly initialized.
Hey, this may be due to me being an idiot, but I'm running into a dependency error when I add Karyon to my project in Maven. The exact error is:
Could not resolve dependencies for project com.trottercashion.karyon-test:karyon-test:jar:0.0.1-SNAPSHOT: Failure to find com.google.code.findbugs:annotations:jar:1.3+ in http://repo.maven.apache.org/maven2
The only change I've made to my pom.xml
from when it was working was adding the following to my <dependencies>
element:
<dependency>
<groupId>com.netflix.karyon</groupId>
<artifactId>karyon-core</artifactId>
<version>1.0.2</version>
</dependency>
<dependency>
<groupId>com.netflix.karyon</groupId>
<artifactId>karyon-admin-web</artifactId>
<version>1.0.2</version>
</dependency>
Any ideas?
We need to initialize archaius to use app + env specific property config
There should be a way inside karyon to be able to do certain custom tasks at certain points in the initialization of karyon. eg: there is no way to do some custom logic between the time configuration is loaded and ServerBootstrap is initiated.
It will be nice to be able to define various phases in the initialization of karyon that the users can intercept and do custom bootstrapping/initialization.
Since, the initialization happens much before governator is initialized & karyon server itself does not get instantiated/managed by governator, governator's lifecycle listeners can not help here.
The helloworld example does not demonstrate what it takes to use jersey and guice together.
This is a question that has been asked by a few people.
Shouldn't each extension be a subproject, rather than having them all in a grab bag together? This way you won't end up dragging in dependencies for extensions that you aren't using.
The Closables.closeQuitely() method has been removed from Guava, (as of 16.0) and causes exceptions during shutdown. It has been replaced with a new method that asks if exceptions should be swallowed.
Reproducible in this project: https://github.com/robertjchristian/karyon-tomcat
Code is identical to current Netflix/karyon, just organized a little differently.
The generated POM for karyon-eureka-2.0.02 in maven central has gradle dependency version syntax.
e.g.
com.netflix.archaius archaius-core 0.+ compilerather than
com.netflix.archaius archaius-core [0,) compileresponse = client.execute(healthGet); can throw HttpHostConnectException because the servlet is not always available when the test is run. I need to get around this quickly, and was able to stopgap-fix with the following sleep hack. Ideally we can guard on "HealthServlet.isReady()" ... alternatively, perhaps the initializer can provide something like isReady() ?
@test
public void testBasic() throws Exception {
startServer();
HttpClient client = new DefaultHttpClient();
HttpGet healthGet = new HttpGet("http://localhost:" + AdminResourcesContainer.LISTEN_PORT_DEFAULT + "/healthcheck");
HttpResponse response = null;
try {
response = client.execute(healthGet);
} catch (HttpHostConnectException e) {
Thread.sleep(1000); // HACK!
try {
response = client.execute(healthGet);
} catch (HttpHostConnectException e2) {
throw e;
}
}
Assert.assertEquals("admin resource health check failed.", 200, response.getStatusLine().getStatusCode());
}
This will provide a lean event based service container for karyon based services.
Hi Nitesh,
Do you have any plans to simplify further to a github project that is simply hello-world and references karyon-core, karyon-extensions, karyon-spi, and karyon-admin as jar/artifact rather than source?
That's how others are likely to want to use this. (IE non-Netflix folks won't/shouldn't need to modify those modules)
Thanks,
Rob
My project is built with Jersey 2.3, when I try to use Karyon admin, I get the error below and the app won't start properly. It seems that it was caused by a conflict between jax-rs api 1 and 2.
(If I keep jsr311-api-1.1.1 or any jar containing jaxrs-1.x API, my Jersey 2 project wouldn't work, and if I keep only javax.ws.rs-api-2.0.jar, Kayron admin wouldn't start. Both version of jax-rs API has javax.ws.rs.core.UriBuilder.class)
Do you guys know of any way to work around this problem?
2013-09-25 13:47:15,764 WARN [qtp791589140-42] o.e.j.s.ServletHandler [ServletHandler.java:539] Error for /admin
java.lang.AbstractMethodError: javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119) ~[javax.ws.rs-api-2.0.jar:2.0]
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:879) ~[jersey-bundle-1.9.1.jar:1.9.1]
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:843) ~[jersey-bundle-1.9.1.jar:1.9.1]
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:804) ~[jersey-bundle-1.9.1.jar:1.9.1]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1300) ~[jetty-servlet-7.6.7.v20120910.jar:7.6.7.v20120910]
at com.netflix.adminresources.RedirectFilter.doFilter(RedirectFilter.java:54) ~[karyon-admin-1.0.20.jar:na]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1300) ~[jetty-servlet-7.6.7.v20120910.jar:7.6.7.v20120910]
at com.netflix.adminresources.LoggingFilter.doFilter(LoggingFilter.java:64) ~[karyon-admin-1.0.20.jar:na]
...
Currently, KaryonBootstrapSuite
configures the HealthCheckHandler
implementation and the only way to override the same is to not use @KaryonBootstrap
and use the KaryonBootstrapSuite
directly.
If we provide an attribute in @KaryonBootstrap
to specify the HealthCheckHandler
implementation, it will make this mainline usecase simpler.
There are multiple properties files for explores included in the admin module. It is unclear why there need to be two and what the difference is.
There are a couple of memory leak fixes since the version listed as a dependency was released.
https://github.com/Netflix/governator/blob/ad60db944b0bec01ecadaf4e4a1cc712c868eb71/CHANGES.txt
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.