Furthermore, even though it is actually an OSGi console plugin, it is Equinox specific, so not all that useful now. The Felix GOGO shell is more popular and will (?) get to be the official specified way to make OSGi console plugins.

DTAP variable

I would like to see a new variable introduced in Navajo. DTAP:
Development
Testing
Acceptance
Production

And make sure they are compiled with the rest

This is incorrect, and possibly (?) dangerous as it can lead to ClassCastExceptions.

Figure out why Maven includes it, and remove it.

The Navajo runtime project com.dexels.navajo.runtime.dependencies includes some binaries;

com.oracle.jdbc.driver_11.2.33.jar
org.apache.felix.bundlerepository-1.6.6.jar
org.apache.felix.eventadmin-1.2.14.jar
org.apache.felix.gogo.command_0.12.0.jar
org.apache.felix.gogo.runtime_0.10.0.jar
org.apache.felix.gogo.shell_0.10.0.jar
org.apache.felix.http.api-2.2.0.jar
org.apache.felix.http.base-2.2.0.jar
org.apache.felix.http.bridge-2.2.0.jar
org.apache.felix.http.bundle-2.2.0.jar
org.apache.felix.http.whiteboard-2.2.0.jar
org.apache.felix.log-1.0.1.jar
org.apache.felix.metatype-1.0.4.jar
org.apache.felix.prefs-1.0.4.jar
org.apache.felix.scr-1.6.0.jar
org.apache.felix.scr.annotations-1.6.0.jar
org.apache.felix.scr.generator-1.1.4.jar
org.apache.felix.webconsole-3.1.8.jar
org.mozilla.javascript_1.7.2.v201005080400.jar
osgi.enterprise.jar

Most are felix dependencies, I need to add them and suppress additional unwanted dependencies by supplying 'provided' scope dependencies.

Two reasons:

• Licence issues (esp. the oracle driver, maybe just move that one to enterprise, for less visibility?)
• It grows the repo too much.

I.e.:

Functions
Adapters
Repositories
Types (?)
Components

The export I've got now only works on the same platform as the feature export was done.

According to stackoverflow it IS possible to create a multi platform feature:

http://stackoverflow.com/questions/10514993/how-can-i-export-an-eclipse-feature-for-multiple-platforms

I just need to figure out how.

Navajo used to rely a lot on split packages (for the record, a split package is a package that exists in multiple jars / bundles)

Split packages can cause a few problems:

OSGi does not like them, as you will (should actually) define dependencies by referencing required packages. OSGi will then look for any bundle that exports that package, and wire the dependency to this package. This is not what you want, as you get a more or less 'random' implementation, instead of a concatenation of all of those packages.

Second: Java classloaders will refuse loading classes from the same package from different jars if the signing information is different. I want to use the same jars throughout (webstart, Eclipse plugins, servlets) so the only way to do that is by signing all bundles.

To fix this:
Actually renaming the packages isn't that tricky any more since we've moved away from CVS, but for functions and adapters it might still break a lot.

• Functions are only a problem in TSL. In navascript they are simply identified by name, and resolved by the function manifest. TSL demands that they are in com.dexels.navajo.functions, I think.
• Adapters Mostly problematic in TSL, as there they are referenced with their fully qualified name, so changing a package will break all scripts using it.

Right now, there is a sizable leak when redeploying. This needs to be addressed.

Needed for further work on the build script of the maven/war deploy

When shutting down a Felix/Navajo instance, Tomcat will show a lot of errors.

Tomcat indicates that it isn't a big deal, and shouldn't be a problem, but it is ugly nonetheless.

I think that the problem is that Tomcat shuts down the webapp classloader before the OSGi runtime has completely stopped

As the bundle state is persistant, it shouldn't be necessary to always do that.

Not doing that should speed up server startup times, secondly, it will reduce the annoying 'State change in progress errors'

Example

org.osgi.framework.BundleException: State change in progress for bundle       "file:/Users/frank/git/demo/navajo/example/com.dexels.navajo.example/adapters/com.dexels.navajo.example.listener_1.0.0.201206261639.jar" by thread "fileinstall-/Users/frank/git/demo/navajo/example/com.dexels.navajo.example/adapters".
at org.eclipse.osgi.framework.internal.core.AbstractBundle.beginStateChange(AbstractBundle.java:1087)
at org.eclipse.osgi.framework.internal.core.PackageAdminImpl.suspendBundle(PackageAdminImpl.java:329)
at org.eclipse.osgi.framework.internal.core.PackageAdminImpl.processDelta(PackageAdminImpl.java:466)
at org.eclipse.osgi.framework.internal.core.PackageAdminImpl.doResolveBundles(PackageAdminImpl.java:251)
at org.eclipse.osgi.framework.internal.core.PackageAdminImpl$1.run(PackageAdminImpl.java:173)
at java.lang.Thread.run(Thread.java:680)

The current target platform uses 3.7.0, which as a bug concerning Felix Webconsole:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=350935

Also, I think Equinox 3.8 uses the Gogo shell, like Felix, so we can use cross platform shell commands.

Runtime works fine, editors are pretty much finished, I just need to link the two.

Dit ter info voor later.
Na wat spontane problemen met opstarten van de clubapp, blijkt dat de combinatie jdk.1_6_30 met ojdbc6 niet goed werkt.
Eigenlijk zelfs nog wat vreemder, want het heeft de hele tijd wel gewerkt, maar ineens niet meer. Update van iets????
Probleem zit 'm in het uitlezen van een BLOB field uit de database. getString op zo'n veld met de waarde null levert een SQLException op.
Om te voorkomen al te ver de diepte in te gaan, heb ik de boel nu weer werkend met de combinatie jdk.1_6_29 en ojdbc14.
Is dus wel iets waar we wat mee moeten in de toekomst.

Create a wiki entry to explain how to build Navajo using Maven

Now, even for small light-weight solutions, we need to supply junit and hamcrest, otherwise it refuses to budge.

Makes no sense for a production environment.

Ideally, we should split off the testing code altogether

• Separate bundles?
• Fragments
• Refactor them as pure 'build code'

.cvsignore, allcvg/pom files, pom_children pom_plugins, git_pom

I'm not sure how that would work though, need to look into that one.

com.dexels.navajo.webconsole.branding is in the 'server' part. This is a bit debatable, it should then be at least com.dexels.navajo.server.webconsole.branding

Or perhaps move it to runtime, in that case, rename the project to com.dexels.navajo.runtime.webconsole.branding

Heb performance van Javascript compiler bekeken. De 'hotspot' (99.9% van de tijd) zit hier:

cx.evaluateReader(globalScope, includeCompiledReader, "includeCompiled.js", 1, null);
cx.evaluateReader(globalScope, includeReader, "include.js", 1, null);
cx.evaluateReader(globalScope, fileReader, a.getRpcName()+ ".js", 1, null);

Met name evalueren van includeCompiled.js kost altijd zo'n 30-40ms... ook voor kleine scriptjes... Volgens mij wordt include.js niet gebruikt.
Is het op een of andere manier niet mogelijk om de includeCompiled.js evaluatie te 'sharen'??

Wat betreft het re-used van includeCompiled.js. Dat is volgens mij mogelijk, ongeveer zoals hieronder.
Dan is het echter wel handig dat de RhinoRunner wordt ge-reused...

Reader includeCompiledReader = getIncludeReader("includeCompiled.js");
Context cx = Context.enter();
includedScope = cx.initStandardObjects();
cx.evaluateReader(includedScope, includeCompiledReader, "includeCompiled.js", 1, null);

    ScriptableObject scope = (ScriptableObject) cx.initStandardObjects(includedScope);

Hier staan ook nog wat performance improvement tips:

http://code.google.com/p/wro4j/wiki/RhinoPerformanceImprovement

TODO

I doubt if we'll use it much internally (I'm more partial to using Nexus, especially when they get their p2 support up to scratch) but it might make sense for clients

Also add to Maven build

Shell commands were never part of the OSGi specification, hence Felix and Equinox built vastly different versions. The version of Apache Felix (called 'gogo') is superior and is now the official standard api for OSGi consoles. From Equinox 3.8 on, Equinox will also support this shell.

We've made a few (3 or 4 I think) OSGi commands for Equinox, we need to port them to Gogo, so they'll work both under Felix and Equinox.

When a Oracle SP is passed a null valued Binary parameter, the call fails.
I think the problem is here (in SPMap.java):

if ( (param == null) || (param instanceof NavajoType && ((NavajoType) param).isEmpty() ) ) {
callStatement.setNull(i + 1, Types.VARCHAR );
}

For Binary it should just perform the setBlob() method, even if its empty.

The build would work, but I've removed them from the total build because I can't get the target platform 'right' for all different platforms.

If I export the jogamp packages on a Mac, the build will work on a Mac, but not on others. I could export it on Linux so the CI works, but that would not help us much. For now, I still hope Nexus will finish it's p2 support so all these problems just go away.

break binnen map geeft wat verwarrende melding....

440525 [KNVBOntwikkelArjen/1337180848720-81] INFO ROOT - Debug: BEFORE BREAK....
440527 [KNVBOntwikkelArjen/1337180848720-81] ERROR com.dexels.navajo.rhino.RhinoRunner - ScriptStack:
at includeCompiled.js:21 (callMap)
at vla/relation/entity/oracle/UpdateOrganizationMember.js:267
at includeCompiled.js:305 (addField)
at vla/relation/entity/ProcessUpdateOrganizationMember.js:297 (anonymous)
at includeCompiled.js:12 (callMap)
at vla/relation/entity/ProcessUpdateOrganizationMember.js:295
at includeCompiled.js:305 (addField)
at vla/syncunion/tounion/ProcessDeleteMemberFunction.js:312 (anonymous)
at includeCompiled.js:12 (callMap)
at vla/syncunion/tounion/ProcessDeleteMemberFunction.js:299 (anonymous)
at includeCompiled.js:222 (forEachMessage)
at vla/syncunion/tounion/ProcessDeleteMemberFunction.js:298

o: org.mozilla.javascript.Undefined@18fd54ec
440527 [KNVBOntwikkelArjen/1337180848720-81] ERROR com.dexels.navajo.rhino.RhinoRunner - Other exception:
org.mozilla.javascript.JavaScriptException: undefined (includeCompiled.js#21)
at org.mozilla.javascript.gen.includeCompiled_js_259._c_callMap_1(includeCompiled.js:21)
at org.mozilla.javascript.gen.includeCompiled_js_259.call(includeCompiled.js)
at org.mozilla.javascript.optimizer.OptRuntime.callName(OptRuntime.java:97)
at org.mozilla.javascript.gen.vla_relation_entity_oracle_UpdateOrganizationMember_js_261._c_script_0(vla/relation/entity/oracle/UpdateOrganizationMember.js:267)
at org.mozilla.javascript.gen.vla_relation_entity_oracle_UpdateOrganizationMember_js_261.call(vla/relation/entity/oracle/UpdateOrganizationMember.js)
at org.mozilla.javascript.gen.vla_relation_entity_oracle_UpdateOrganizationMember_js_261.exec(vla/relation/entity/oracle/UpdateOrganizationMember.js)
at org.mozilla.javascript.Context.evaluateReader(Context.java:1142)
at com.dexels.navajo.rhino.RhinoRunner.runScript(RhinoRunner.java:241)
at com.dexels.navajo.rhino.RhinoRunner.run(RhinoRunner.java:368)
at com.dexels.navajo.rhino.RhinoCompiledScript.execute(RhinoCompiledScript.java:38)
at com.dexels.navajo.mapping.CompiledScript.run(CompiledScript.java:282)
at com.dexels.navajo.rhino.RhinoHandler.doService(RhinoHandler.java:38)
at com.dexels.navajo.server.ServiceHandler.construct(ServiceHandler.java:63)
at com.dexels.navajo.persistence.impl.PersistenceManagerImpl.get(PersistenceManagerImpl.java:150)
at com.dexels.navajo.server.Dispatcher.dispatch(Dispatcher.java:445)
at com.dexels.navajo.server.Dispatcher.processNavajo(Dispatcher.java:1054)

Have a look at joda:

http://joda-time.sourceforge.net/index.html

Perhaps we can replace some of our own code with this in respect to the time and money types

asasa

We only use package imports anyway (of those select few bundles), so we should be able to do this transparently.
This will release us of the strict dependency com.dexels.navajo.client has on the jabber libraries (smack and smackx)

Implement a run-time mechanism to detect infinite loops in navajomap calls to the same webservice...
Could be subtle because a call to same webservice might be valid if a different branch of the script is followed...
Somehow Navajo Runtime should be able to detect loops...

We can run now from eclipse, we need a deployable war file to be deployed into a regular tomcat environment

Once the Navajo Maven / OSGi build is up and running, create a similar (but much simpler) build for the Tipi app store.

First, the java code needs to be in its own project, after that a regular JSP based webapp.

Java based scripts can be made to work in an OSGi setting. Problem is, due to the type of code generated by the Tsl>Java compiler, that it needs class access to the adapters it uses.

This means that, just like it is without OSGi, that every script needs its own class loader. The logical way to do this in OSGi would be to generate a separate bundle for every script. At first I thought that would be ridiculous, but I'm starting to change my mind.

The things we do (classloader for each script) and the things we want to do (proactive dependency management) just covers that area. We made our own custom classloader which has worked fine, so why wouldn't it work in OSGi?

Advantages:

• We can use declarative services to manage our dependencies
• We have a very clean interface to create multi lingual scripts (A 'script', at runtime level, is a bundle that can import whatever it wants, and provides a service of type CompiledScript)
• It pretty agnostic of languages, it will work equally well with any OSGi-capable JVM language.
• It simplifies the tasks of the dispatcher
• We can use the versioning capabilities of OSGi (by allowing multiple versions of the same bundle), the present '_beta' suffix.

Challenges:

• The number of bundles is going to be really large, 5000 bundles, with 5000 services and who knows how many dependencies, I fear for memory usage.
• OSGi tools like the webconsole or regular console commands might not be suited for this number of bundles.
• I don't know how fast the dependency management is at this scale.
• Getting the java compiler to work might be tricky
• It will only work in OSGi

Allow the script to supply a binary certificate to use in the HTTPs connection.

I think the biggest challenge is to setup a test environment.

They do look a bit stupid. I've left them unchanged until I can do them all, as they are easier to find this way

The bundle name is com.dexels.navajo.tipi.charting, should be com.dexels.navajo.tipi.swing.charting

There are a few ant tasks that Tipi relies on to build proper tipi extensions.

Note that right now, I don't think any implementations rely on it, but I do think it still makes sense to keep this code in business.

Give some hints about how to use the Sonatype Nexus server at Dexels

Wat te doen met de ResourceChecker klasse. Deze heeft dependency naar GenericHandler en dus naar TslCompiler. Hierin worden de 'dependencies'
van een script bepaald, zoals includes, navajomaps, adapters etc..
Op dit moment worden dus een .js een .java en een .class file gegenereerd bij script compile.

The unit tests are run with the old Ant-based build scripts, but they haven't been included yet in the Maven build

Looking like:

HTTP ERROR 500

Problem accessing /system/console/components. Reason:

INTERNAL_SERVER_ERROR

Caused by:

java.lang.NullPointerException
at org.eclipse.equinox.internal.ds.impl.ReadOnlyDictionary.get(ReadOnlyDictionary.java:86)

The eclipse bug is here:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=350935

It's already fixed in the latest maintenance release 3.7.2, I am using 3.7.0, which still has this bug

This is an unnecessary dependency, if we use DS we can just use a Pojo.

DS will manage navajo clients if DS is present, if not, we don't have a stale OSGi api dep.

Specifically:

• Generate wiki code from Maven
• Generate GitHub Markdown compatible code

In the Navajo scheme of things, lazy activation is pretty much always a bad idea, and result in unexpected behaviour. So remove it in all bundles, also internal and enterprise.