fnproject / fdk-java Goto Github PK

The Java SE docs list a number of troubleshooting tools: the debugger, JFR, various profiling tools. We might bring any
number of these to bear. I think there are a few axes we can look at: local / deployed in the cloud; live / snapshot /
post-hoc; "plain" java fdk and a flow invocation; and using a cooperative container versus an unadorned one
(together with soem kind of sidecar).

There are sufficient reasons why we might want to extend the debugging capabilities to a cloud-deployed function:
they may be plumbed in against different persistence apps; we might have connectivity trouble; we might have strange
behaviour with a particular client not seen in local testing, etc.

Live debugging, locally / in the cloud

The scenario here may be complicated because a developer's machine is sitting behind a firewall/proxy. Additionally,
we don't necessarily want any Tom, Dick or harry to be able to connect a debugger to our running images (the images
might be proprietary), so we'll need this to be authenticated somehow.

On-demand debugging with a cooperative instance, single function call

This is relatively doable - modulo the restriction on an instance lifetime, it can basically be put together without
much help from the functions platform - assuming that functions have reasonable outgoing network connectivity.

• I deploy my app configured (amongst other things) with a key that'll let the instance unlock the debug connection
  details.

• I launch my debugger. For a Java debugger, it's possible to set the JVM to make an outbound connection to a listening
  debugger; this really simplifies this setup.

• If I'm on the internet, I'm basically done. If I'm behind a proxy server, however, I need to make a tunnel connection
  to some bouncer that'll forward a connection to my debugger locally. The tunnel server sends me details of how the
  JVM should connect to it (endpoint address, credentials) and waits.

• I make a customised call to the functions server to invoke my function. I pass additional information - the endpoint
  connection details, encoded so that only my app can decode them with its configured debug key.

• On launch, the presence of the debug header causes the container's entrypoint to extract the connection details and
  launch the JVM with the debug agent loaded. It connects via the supplied address to the bouncer which forwards the
  connection to my debugger. Bob's your uncle.

(With a little more sleight-of-hand, debuggers that require incoming connections to the debugged process could be
managed also; it's a matter of configuring the tunnel/bouncer service correctly.)

For a hot function, we'd need to ensure that the debugger was correctly turned off / disabled (or the function instance
exited after debug) so that other requests that are routed to it don't hit breakpoints that've been left behind.

How this might be improved with assistance from the functions platform

• being able to selectively turn off process timeouts

• we can potentially attach a process debugger to a local PID; this has a bunch of downsides (it's read-only and
  freezes the target process whilst it's connected). A sidecar debugger that can relay connections from a desktop
  tool might be able to do this with access to the process space.

What this feels like

This is basically a traditional debugging scenario.

Flow

There's a major issue with this, and that's that reinvocations of the function are effectively forks; there are new
processes. Whilst we might collaborate with the flow-service in order to deliver the right headers, there are two
main problems:

• firstly, we'd ideally like breakpoints set in the first invocation of a function to persist as far as
  our debugger view is concerned. We might be able to get away with another "suspend on launch" or similar.

• secondly - this may well be more of a barrier to usability - each Flow execution is its own process. Do we
  launch multiple user debugger instances? Multiple processes may be running at once. (How do IDE debuggers cope with
  forking processes, if at all?)

  Possibly one short-term approach here would be to fire up half a dozen (? or so) debuggers each awaiting an incoming
  connection, and each listening on a different channel. The bouncer/tunnel would need to target a new debugger instance
  for each incoming connection. This might prove unusable from a user perspective - would need to experiment.

• One option here is having a user tool which can collaborate and knows about the Cloud Threads architecture: on a new
  invocation, any break-points etc. are stored and that configuration is retained. For additional debug connections that
  come in from new cloud futures, that debug configuration is restored to the new future before it is run.

  The bouncer/tunnel could potentially assist with this by intercepting debugger traffic and inspecting it, keeping some
  picture of the desired state and relaying it to new instances.

Snapshots

The idea here is that the function is pre-deployed and presumably under some kind of user-driven load.

• The user asks for a snapshot. This might be down to a number of criteria (breakpoints, other conditions).

• Additional configuration is supplied / available to an instance. On launch (possibly: later, for hot functions? This
  would definitely require more assistance from the functions platform to achieve) a function can be configured to
  start up a debugging agent (note, this can potentially be done with the stock agent). A nearby service (potentially,
  in-container, with a cooperative image) connects to this and supplies appropriate breakpoints / watch conditions.

• This setup continues as long as the snapshot request is live. Once a hit is made, the debugger needs to extract
  salient information and send it via a side-channel to its target repository.

• For one-shot shapshots, the condition is then marked as no longer live. If we get more than one hit, one snapshot wins
  (or we collect a bunch of them) but once the trigger is marked as done, future function invocations will not set up
  the breakpoint condition.

This approach feels quite "cloudy"; it's really appealing. It needs a nearby, fast source of data that the container
(resp, the functions platform) itself can configure. It needs a way to rapidly shuttle a snapshot result off the
container and into a bucket for later perusal. For an in-container (cooperative) situation, we'd need to ensure that the
data is delivered before the container shuts down.

If functions are primarily running "hot" (ie, in relatively long-lived containers) then we may need a way to know where
those containers are and to configure them after they have been initially set up. The functions platform would need to
cooperate there ("signalling" debug hooks that new configuration is available). Alternatively, every hot container (or
a fraction of them) could collaborate with a fast message queue if we were rolling this as something that didn't rely
on the functions platform for support.

Pushing config: With some kind of nearby sidecar that's able to attach to the debugger, we'd still want to know which
functions are deployed to our local host. We'd need to subscribe to a topic that supplied debug information for them.
It'd be helpful if functions had some kind of locality to avoid having to know about every single snapshot
request. This might be a more efficient architecture but seems potentially a great deal more complicated.

Given a message bus which we can rapid-fire messages into, similar uses of the same facility (eg, the "ad hoc logging"
facility) are variations on this theme.

The second major consideration is what the API for this looks like. We need to be able to get a bunch of requirements
from the user (file / line number and condition, at the least) into the platform.

Flow

Ironically, there's practically no additional difficulty in the situation where we're doing live snapshots of a
cloud-thread invocation: every function call is treated the same. The main technical barrier here is identifying the
right breakpoint spots - this might be complicated by the use of lambdas. Fiddly technical details abound.

Try to get this ready in time for SpringOne.

• This needs to be a separate jar
• Need to be able to supply a FunctionInvoker from a separate jar
• SCF support
• Example Project
• Blog post

At this moment most of the officially supported FDKs have a different understanding of what the context is. The bigger difference is in the context API. This issue is all about standardizing the context across FDKs that Fn officially supports.

By default, the generated class is HelloFunction.java and the artifact is

    <groupId>com.example.fn</groupId>
    <artifactId>hello</artifactId>

Using the function name for the artifactId and the class name might be a more sensible default value.

Users should be able to specify their own Invoker (or add it to the invoke chain) to allow native internal access to FDK events.

it's ugly to have to construct your own strings from a byte array.

I want to be able to verify the inputs to a given InvokeFunction stage in the graph in test.

I want to be able to :

   testing.getOnlyCall("./testFn") -> some kind of http request| exception if there isn't exactly one call 

   testing.getAllCalls("./testFn") - list of call objects. 

If I do a toString on the result of invokeFunction i get an anonymous class From SerUtils - this shuold be a concrete class with a sensible toString.

I'm attempting to duplicate the FN flow "first flow" tutorial as described here: https://github.com/fnproject/fdk-java/blob/master/docs/FnFlowsUserGuide.md.

My project follows the "primes" example pretty closely in structure and code:

https://imgur.com/a/3TMdQ

And here are my command line operations;

patientplatypus:~/Documents/Oracle/DragnDropInfra/fn_flows/testflow:14:37:31$fn apps create flows-example10
Successfully created app:  flows-example10
patientplatypus:~/Documents/Oracle/DragnDropInfra/fn_flows/testflow:14:38:47$fn deploy --app flows-example10
Deploying patientplatypus/flow-primes to app: flows-example10 at path: /primes
Bumped to version 0.0.14
Building image patientplatypus/flow-primes:0.0.14 
Pushing patientplatypus/flow-primes:0.0.14 to docker registry...The push refers to repository [docker.io/patientplatypus/flow-primes]
505f240bdfa8: Layer already exists 
895a2a3582de: Layer already exists 
5fb388f17d37: Layer already exists 
c5e4fcfb11b0: Layer already exists 
ae882186dfca: Layer already exists 
aaf375487746: Layer already exists 
51980d95baf3: Layer already exists 
0416abcc3238: Layer already exists 
0.0.14: digest: sha256:02d33692c43c7b49387078b82c43812e08841ec9234a944bcd4d7d46e5a483a7 size: 1997
Updating route /primes using image patientplatypus/flow-primes:0.0.14...
patientplatypus:~/Documents/Oracle/DragnDropInfra/fn_flows/testflow:14:39:02$fn apps config set flows-example10 COMPLETER_BASE_URL "http://localhost:8081"
flows-example10 updated COMPLETER_BASE_URL with http://localhost:8081
patientplatypus:~/Documents/Oracle/DragnDropInfra/fn_flows/testflow:14:39:25$echo 10 | fn call flows-example /primes
Error: Route not found
ERROR: error calling function: status 404
patientplatypus:~/Documents/Oracle/DragnDropInfra/fn_flows/testflow:14:39:34$curl -XPOST -d "10" http://localhost:8080/r/flows-example10/primes
patientplatypus:~/Documents/Oracle/DragnDropInfra/fn_flows/testflow:14:40:49$curl -XPOST -d "10" http://localhost:8080/r/flows-example10/primes
patientplatypus:~/Documents/Oracle/DragnDropInfra/fn_flows/testflow:14:40:57$curl -XPOST -d "10" http://localhost:8080/r/flows-example10/primes --verbose
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> POST /r/flows-example10/primes HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Length: 2
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 2 out of 2 bytes
< HTTP/1.1 500 Internal Server Error
< Content-Length: 0
< Content-Type: text/plain; charset=utf-8
< Fn_call_id: 01C9SMEAG9NG8G00RZJ0000013
< Xxx-Fxlb-Wait: 78.389µs
< Date: Thu, 29 Mar 2018 19:41:06 GMT
< 
* Connection #0 to host localhost left intact
patientplatypus:~/Documents/Oracle/DragnDropInfra/fn_flows/testflow:14:41:04$

As you can see I don't include the line DOCKER_LOCALHOST=$(docker inspect --type container -f '{{.NetworkSettings.Gateway}}' functions), because it fails when I try and run it. If functions is a stand in for something please include that in the documentation, I'm not sure what you're referring to.

patientplatypus:~/Documents/Oracle/DragnDropInfra/fn_flows/testflow:14:41:04$DOCKER_LOCALHOST=$(docker inspect --type container -f '{{.NetworkSettings.Gateway}}' functions)
Error: No such container: functions

However, I did specify that it should be ok to run this on localhost and I still get a 500 error. Would it be possible to point me to a git repo of this example working or let me know what I'm doing wrong? Feel free to email me at [email protected] or [email protected]. Thanks.

This issue is created to track the movement of any Java examples or Java FDK users docs that are being moved into the docs repo.

This issue will be closed once those files have been successfully moved to Docs and deleted from this Repo.

Hot Java functions can only be created using the HTTP format/container-protocol, which should be deprecated in favour of JSON format. But that can't happen until the Java FDK supports the JSON protocol between Fn Server and the function container...

Docker uses: http://tldp.org/LDP/abs/html/exitcodes.html
Make sure FDK does not allow functions to exit with these codes.
For example, 137 exit code is currently flagged as OOM in fn-agent, therefore functions should not be allowed to exit with 137 themselves.

We set our JUnit version to 4.12 here https://github.com/fnproject/fdk-java/blob/master/pom.xml#L31

Typically doing that is fine because junit is normally scoped to testing, and users can be free to choose their version as ours won't be included transitively into their builds. However we have a testing module which has junit as a normally-scoped dep: https://github.com/fnproject/fdk-java/blob/master/testing/pom.xml#L25-L29

We should support users wanting to use JUnit 5, and we should also track JUnit releases.

Currently, a simple Java Hello world function has 2.2mb of jars in the classpath in which almost 2mb are for optional thirdparty libraries.

1.5mb for Jackson Databind
316kb for Commons HTTP core.

Jackson Databind can be completely removed if we use JSON-B standard API in #98, so the user can choose to add the JSON-B implementation of choise if he needs to support JSON coersion or JSON events format.

Commons HTTP can be optional if user needs to support func.yaml http format.
I think in future this format should be deprecated in favor of Default, JSON & Cloud Events

This improvement also need #133 to be handled.

An automated process for...

Minimal (smithed?) images for runtime.
Oracle linux based for build images.

After the adding JSON support for Java Fn function, if the function is passed an empty value, Fn returns:

{"message":"container exit code 2"}

Fn: Error calling function: status 502

The function does work with JSON data. If you pass:

{"name":"Bob"}

or even

{"name":""}

the function works fine. Tried modifying the function to test for null and isEmpty. Nothing like that worked. I'm guessing the function is throwing an uncaught exception.

May be related to 392 or 1192

Mike W.

I want to be able to use JSON-B as an alternative to Jackson for JSON marshalling

Is one or other included by default? Can i switch between marshalling by adding a module to my function?

The readme.md uses commands that don't exist in v2 like "fn run" or "fn call". This Readme is linked in docs as the tutorial for getting started with the Java FDK, so it's very likely the first thing Java devs try.

Whenever I want to write a function which depends on a jar that isn't in a public maven repo it's a pain. I end up creating a custom Dockerfile with something like:

COPY deps /usr/share/maven/ref/repository

to pre-seed the in-image maven repo cache with stuff I am working on right now. This could actually be in the boilerplate, no?

This happens for the boilerplate-only java project ,during fn build.

Step 5/11 : RUN ["mvn", "package", "dependency:copy-dependencies", "-DincludeScope=runtime", "-DskipTests=true", "-Dmdep.prependGroupId=true", "-DoutputDirectory=target", "--fail-never"]
 ---> Running in 29e9543024d0
Error: Could not find or load main class 127.0.0.0.8|
Caused by: java.lang.ClassNotFoundException: 127.0.0.0.8|

Any idea why it mistakes 127.0.0.8 as a class name? Anything to do with the proxy?
Thanks

The examples/async-thumbnails/pom.xml specifies the fnproject.version as 1.0.0-SNAPSHOT, which causes the build to fail because that version does not exist in the fn-maven-releases repository. Removing the -SNAPSHOT should solve the problem.

All published fdk-java artifacts that are intended to be used by others should set a stable automatic module that doesn't rely on the file name of each artifact. For more details, see for example https://blog.joda.org/2017/04/java-se-9-jpms-module-naming.html

Available Maven Modules

Copied from https://github.com/fnproject/fdk-java/blob/master/pom.xml#L13-L24

    <modules>
        <module>api</module>
        <module>runtime</module>
        <module>testing-core</module>
        <module>testing-junit4</module>
        <module>testing</module>
        <module>flow-api</module>
        <module>flow-runtime</module>
        <module>flow-testing</module>
        <module>fn-spring-cloud-function</module>
        <module>examples</module>
    </modules>

Maven Module (and JAR name) - Java Module Name

• api[.jar] - com.fnproject.fn.api
• ...

Hi,

I tried to run a java function in fn, but I got the following error

$sudo fn run
Building image xxx/xxx:0.0.10
Sending build context to Docker daemon  72.19kB
Step 1/11 : FROM fnproject/fn-java-fdk-build:jdk9-latest as build-stage
jdk9-latest: Pulling from fnproject/fn-java-fdk-build
3f84388a3335: Already exists
5af7807a0ec3: Already exists
09d580f15c0b: Already exists
23a3b03fc984: Already exists
08022c1872b4: Already exists
ea733b0154e4: Pull complete
eb337c138275: Pull complete
713f0a7fd50b: Pull complete
0d5d9b25d97b: Pull complete
b6d0604a0963: Pull complete
a1faa3b40025: Pull complete
62988b881ae7: Pull complete
a117f344a64f: Pull complete
6f7ed3e6370c: Pull complete
Digest: sha256:13ef2b43fce6cb13c15993f6baa159ebc4b207a2429b247e07089fe9ef8fd9ab
Status: Downloaded newer image for fnproject/fn-java-fdk-build:jdk9-latest
 ---> f9d235fdddf7
Step 2/11 : WORKDIR /function
 ---> aad0e2e2c2be
Removing intermediate container 8018a01ad48c
Step 3/11 : ENV MAVEN_OPTS -Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort= -Dhttp.nonProxyHosts= -Dmaven.repo.local=/usr/share/maven/ref/repository
 ---> Running in 6d3948c5ff0b
 ---> 3d4ef0541bfe
Removing intermediate container 6d3948c5ff0b
Step 4/11 : ADD pom.xml /function/pom.xml
 ---> bf7c5e14fb0d
Step 5/11 : RUN mvn package dependency:copy-dependencies -DincludeScope=runtime -DskipTests=true -Dmdep.prependGroupId=true -DoutputDirectory=target --fail-never
 ---> Running in 628bf9f836b8
/docker-java-home/bin/java: error while loading shared libraries: libjli.so: cannot open shared object file: No such file or directory
The command 'mvn package dependency:copy-dependencies -DincludeScope=runtime -DskipTests=true -Dmdep.prependGroupId=true -DoutputDirectory=target --fail-never' returned a non-zero code: 127
ERROR: error running docker build: exit status 127

The images that I've downloaded:

$sudo docker images
REPOSITORY                                           TAG                 IMAGE ID            CREATED             SIZE
<none>                                               <none>              bf7c5e14fb0d        19 minutes ago      407MB
fnproject/fn-java-fdk-build                          jdk9-latest         f9d235fdddf7        6 hours ago         407MB
fnproject/functions                                  latest              839b553dbbe7        2 days ago          143MB

When you dig into the fnproject/fn-java-fdk-build:jdk9-latest, actually I've found that the same error.

$sudo docker run -ti fnproject/fn-java-fdk-build:jdk9-latest /bin/bash
root@a7c1451c7413:/# ls
bin   dev               etc   lib    lib64   media  opt   root  sbin  sys  usr
boot  docker-java-home  home  lib32  libx32  mnt    proc  run   srv   tmp  var
root@a7c1451c7413:/# /docker-java-home/bin/java
/docker-java-home/bin/java: error while loading shared libraries: libjli.so: cannot open shared object file: No such file or directory
root@a7c1451c7413:/#

I think there is something wrong with the image.

https://github.com/fnproject/fdk-java#3-run-your-first-java-function

Don't think it is?

I think this might be related to some of the mount stuff that the maven image is doing but:

opc@occ-test jfn]$ fn build  --no-cache
Building image jfn:0.0.2
Sending build context to Docker daemon  93.18kB
Step 1/11 : FROM fnproject/fn-java-fdk-build:jdk9-latest as build-stage
 ---> cc6e2559930d
Step 2/11 : WORKDIR /function
 ---> 5f2ca164bb57
Removing intermediate container d8a68d088107
Step 3/11 : ENV MAVEN_OPTS -Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort= -Dhttp.nonProxyHosts= -Dmaven.repo.local=/usr/share/maven/ref/repository
 ---> Running in 47edd2bc8261
 ---> 4427a9bd1b18
Removing intermediate container 47edd2bc8261
Step 4/11 : ADD pom.xml /function/pom.xml
 ---> b8fdf9909525
Removing intermediate container 5981a4093567
Step 5/11 : RUN mvn package dependency:copy-dependencies -DincludeScope=runtime -DskipTests=true -Dmdep.prependGroupId=true -DoutputDirectory=target --fail-never
 ---> Running in 4c18883d065f
[INFO] Scanning for projects...
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-deploy-plugin/2.7/maven-deploy-plugin-2.7.pom
[WARNING] Failed to create parent directories for tracking file /usr/share/maven/ref/repository/org/apache/maven/plugins/maven-deploy-plugin/2.7/maven-deploy-plugin-2.7.pom.lastUpdated
[WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-deploy-plugin:2.7: Plugin org.apache.maven.plugins:maven-deploy-plugin:2.7 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-deploy-plugin:jar:2.7
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building hello 1.0.0
[INFO] ------------------------------------------------------------------------
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-deploy-plugin/2.7/maven-deploy-plugin-2.7.pom
[WARNING] Failed to create parent directories for tracking file /usr/share/maven/ref/repository/org/apache/maven/plugins/maven-deploy-plugin/2.7/maven-deploy-plugin-2.7.pom.lastUpdated
[WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-deploy-plugin:2.7: Plugin org.apache.maven.plugins:maven-deploy-plugin:2.7 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-deploy-plugin:jar:2.7
Downloading from fn-release-repo: https://dl.bintray.com/fnproject/fnproject/com/fnproject/fn/api/1.0.48/api-1.0.48.pom
Downloaded from fn-release-repo: https://dl.bintray.com/fnproject/fnproject/com/fnproject/fn/api/1.0.48/api-1.0.48.pom (0 B at 0 B/s)
[WARNING] Failed to write tracking file /usr/share/maven/ref/repository/com/fnproject/fn/api/1.0.48/_remote.repositories
java.io.FileNotFoundException: /usr/share/maven/ref/repository/com/fnproject/fn/api/1.0.48/_remote.repositories (Permission denied)
    at java.io.RandomAccessFile.open0 (Native Method)
    at java.io.RandomAccessFile.open (RandomAccessFile.java:343)
    at java.io.RandomAccessFile.<init> (RandomAccessFile.java:259)
    at java.io.RandomAccessFile.<init> (RandomAccessFile.java:214)
    at org.eclipse.aether.internal.impl.TrackingFileManager.update (TrackingFileManager.java:105)
    at org.eclipse.aether.internal.impl.EnhancedLocalRepositoryManager.addRepo (EnhancedLocalRepositoryManager.java:193)
    at org.eclipse.aether.internal.impl.EnhancedLocalRepositoryManager.addArtifact (EnhancedLocalRepositoryManager.java:171)
    at org.eclipse.aether.internal.impl.EnhancedLocalRepositoryManager.add (EnhancedLocalRepositoryManager.java:149)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.evaluateDownloads (DefaultArtifactResolver.java:602)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads (DefaultArtifactResolver.java:513)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve (DefaultArtifactResolver.java:399)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts (DefaultArtifactResolver.java:224)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact (DefaultArtifactResolver.java:201)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom (DefaultArtifactDescriptorReader.java:261)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor (DefaultArtifactDescriptorReader.java:192)
    at org.eclipse.aether.internal.impl.DefaultDependencyCollector.resolveCachedArtifactDescriptor (DefaultDependencyCollector.java:539)
    at org.eclipse.aether.internal.impl.DefaultDependencyCollector.getArtifactDescriptorResult (DefaultDependencyCollector.java:522)
    at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:411)
    at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process (DefaultDependencyCollector.java:353)
    at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies (DefaultDependencyCollector.java:256)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies (DefaultRepositorySystem.java:282)
    at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve (DefaultProjectDependenciesResolver.java:169)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:223)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:145)
    at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:246)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:200)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)

Hello; using JDK 9 and Maven 3.5.0 I've:

• cloned the repository
• run mvn -U clean install on it

It fails trying to download the com.github.kenglxn.qrgen:javase:jar:2.3.0 dependency:

[INFO] ------------------------------------------------------------------------
[INFO] Building qr-code 1.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: http://localhost:8081/repository/maven-public/com/github/kenglxn/qrgen/javase/2.3.0/javase-2.3.0.pom
[WARNING] The POM for com.github.kenglxn.qrgen:javase:jar:2.3.0 is missing, no dependency information available
Downloading: http://localhost:8081/repository/maven-public/com/github/kenglxn/qrgen/javase/2.3.0/javase-2.3.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] fdk ................................................ SUCCESS [  2.043 s]
[INFO] api ................................................ SUCCESS [  8.005 s]
[INFO] runtime ............................................ SUCCESS [ 36.625 s]
[INFO] testing ............................................ SUCCESS [ 15.660 s]
[INFO] string-reverse ..................................... SUCCESS [  1.574 s]
[INFO] regex-query ........................................ SUCCESS [  0.247 s]
[INFO] qr-code ............................................ FAILURE [  1.085 s]
[INFO] async-thumbnails ................................... SKIPPED
[INFO] fdk-examples ....................................... SKIPPED
[INFO] integration-tests .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:05 min
[INFO] Finished at: 2017-10-19T09:35:39-07:00
[INFO] Final Memory: 42M/669M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project qr-code: Could not resolve dependencies for project com.fnproject.fn.examples:qr-code:jar:1.0.0-SNAPSHOT: Could not find artifact com.github.kenglxn.qrgen:javase:jar:2.3.0 in nexus (http://localhost:8081/repository/maven-public/) -> [Help 1]

(My Nexus instance running on localhost works fine (i.e. is not the source of the problem), and indeed search.maven.org shows no listing for com.github.kenglxn.qrgen:javase:jar:2.3.0, but does show some other qrgen artifacts. Perhaps one of those is the dependency that should have been included?)

We are planning to deprecate all formats except for the unix-socket based http-stream format in the coming months

for standard JVM functions this requires some JNI to create unix sockets for substrate the functions either need to be linked against this (possibly by compiling the unix socket library against a different target)

https://github.com/fnproject/fn-java-fdk/blob/master/docs/CloudThreadsUserGuide.md

Given a class under test, when FnTesting.thenRun(cls, method) is run where method doesn't exist on cls, the test will fail with

java.lang.IllegalStateException: One and only one response expected, but 0 responses were generated..

Whilst true, this is misleading, we should catch this earlier in the test framework and let the user know that the desired method doesn't exist.

The fn Hot contract supports chunked encoding for output streams - currently the FDK buffers the output to get content length - we should change this to use a chunked transfer to prevent having to buffer the output into the heap of each function.

Actual behaviour:

Function that produces a lot of data will fill up ram of function container.

Expected behaviour:

Function can generate more output than its heap.

Hello people,

making a simple Hello-World as shown in the README, running the "fn run" command gives me the following exception:

Exception in thread "main" java.lang.NoSuchMethodError: com.fnproject.fn.api.InputCoercion.tryCoerceParam(Lcom/fnproject/fn/api/InvocationContext;ILcom/fnproject/fn/api/InputEvent;)Ljava/util/Optional;
	at com.fnproject.fn.runtime.MethodFunctionInvoker.lambda$doInputCoercions$0(MethodFunctionInvoker.java:64)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1471)
	at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:127)
	at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:502)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:476)
	at com.fnproject.fn.runtime.MethodFunctionInvoker.doInputCoercions(MethodFunctionInvoker.java:66)
	at com.fnproject.fn.runtime.MethodFunctionInvoker.tryInvoke(MethodFunctionInvoker.java:34)
	at com.fnproject.fn.runtime.EntryPoint.run(EntryPoint.java:87)
	at com.fnproject.fn.runtime.EntryPoint.main(EntryPoint.java:27)
ERROR: exit status 1

this error appears when I am using within pom.xml at <fdk.version> 1.0.54 </ fdk.version> at 1.0.52.

When I use version 1.0.51 it works normally ,

Javadoc still uses the 'old' functionId pattern, they should be updated to reflect the 'new' functionId style.

E.g.

• fdk-java/flow-api/src/main/java/com/fnproject/fn/api/flow/Flow.java

  Line 34 in 0b7d2e6

  * Function IDs should be of the form "APPID/path/in/app" (without leading slash) where APPID may either be a named application or ".", indicating the appID of the current (calling) function.

• fdk-java/flow-api/src/main/java/com/fnproject/fn/api/flow/Flow.java

  Line 80 in 0b7d2e6

  * @param functionId Function ID of function to invoke - this should be the function ID returned by `fn inspect function appName fnName`

• fdk-java/flow-api/src/main/java/com/fnproject/fn/api/flow/Flow.java

  Line 201 in 0b7d2e6

  * return rt.invokeAsync("testapp/testfn","input".getBytes()).thenApply(String::new);

• fdk-java/flow-api/src/main/java/com/fnproject/fn/api/flow/Flow.java

  Line 223 in 0b7d2e6

  * return rt.invokeAsync("testapp/testfn","input".getBytes()).thenApply(String::new);

• ...

There are several cases where we throw a FunctionInputHandlingException on valid user input:

• The function format is invalid (not default or http), which is a configuration error
• We can't find an invoker that can run the function, which is a programming error
• We have an IOException reading the input stream, which is likely a programming error (read after close)

At the same time, we don't have an exception for a genuine input validation issue.

So we need to:

• separate programming/configuration issues from i/o issues, and use the right exceptions for those
• create a new exception to represent genuinely wrong input

Just noticed the pom file had this in it:

<dependency>
            <groupId>com.fnproject.fn</groupId>

Should probably be io.fnproject.xxx?

The build image has a pre-cached version of some maven modules but it looks like boilerplate generates different plugin dep versions - specifcally for mvn compiler plugin and mvn deploy plugin t

Impact: Longer build times

Considering the following function class

public class PojoFunction {
    public Object greet(String name) {
        if (name == null || name.isEmpty()) {
            name = "World";
        }
        return new Greeting(name);
    }
    public static class Greeting {
        public final String name;
        public Greeting(String name) {
            this.name = name;
        }
    }
}

and this content for test.json

{
    "tests": [
        {
            "input": {"body": "Johnny"},
            "output": {"body": {"name": "Johnny"}}
        }
    ]
}

causes a test failure.

Test fails due to the String value in the greet function being of the value "Johnny" instead of value Johnny.

Console output:

FAILED           mismatched output found.
                 expected:
                 {"name":"Johnny"}
                 got:
                 HTTP/1.1 200 INVOKED
                 Content-Type: application/json
                 Content-length: 21
                 
                 {"name":"\"Johnny\""}

$ fn init --runtime=java your_dockerhub_account/flow-primes

is apparently missing the --name parameter and the command in its current form thus creates some unwanted sub-directories.

The wanted command is probably

$ fn init --runtime=java --name your_dockerhub_account/flow-primes

given: func.yaml

version: 0.0.5
runtime: java
cmd: com.example.fn.PojoFunction::greet
build_image: fnproject/fn-java-fdk-build:jdk9-1.0.56
run_image: fnproject/fn-java-fdk:jdk9-1.0.56
memory: 32
format: http
config:
GREETING: "yes"

When & What

When I deploy this function (fn deploy --app java-app --local) it works as expected, but when I execute it (fn build && fn run ) no config attributes are available.
I checked the map with the configuration, and what I see is:
attributes like "memory" are correctly passed, but no single key/value from config: section. When using UnitTest and passing configuration by using setConfiguration, it works fine

I had a func.yaml with the follwoing content:

name: mail
version: 0.0.3
runtime: java
cmd: com.espirit.fsi.example.ApplicationViaMail.java::handleRequest
build_image: fnproject/fn-java-fdk-build:jdk9-1.0.56
run_image: fnproject/fn-java-fdk:jdk9-1.0.56
format: http

an fn deploy, as well as fn test did not fail, however the docker image was not able to run because of the copy-paste-error in cmd, which contained the .java file extension which was wrong.

My expectation is that an fn test or fn deploy fails when it cannot run properly. One possible solution is to run the docker image locally before deploying it, although it may be possible to check the func.yaml direclty for correctness.

We don't start the JVM with the headless flag - this means that a bunch of AWT operations don't work - should be as simple as adding -Djava.awt.headless=true to the jvm startup in the function runtime image.

https://www.oracle.com/technetwork/articles/javase/headless-136834.html

Currently all function classes are required to have a method which is designated as "the function". This isn't necessary for all invokers, eg those that can determine the method themselves by reflection or other means.

This check should be lifted from the EntryPoint class into the MethodInvoker.

Java SE 11 is a LTS release and will be GA'ed on Sep. 25. We need to make sure Fn support SE 11 at GA time. Moreover, SE 11 should then be the default version.

Fn Build for FDK currently only works with Maven - intention is to support gradle one way or another too.

• Boilerplate
• Build (Code & Container)

Hi, i found that there always be a image named none(tag is also named none) when i deploy a java project successfully, is this normal?

It appeared during a deploy --local --app japp, with no further information

This shouldn't happen (tm) but there are a few places where it's technically possible for types to get mixed up (including around updates etc).

When datum object can't be assigned to the receiver type you get something like the error below.

This should be clearer as it's very hard to work out where this is coming from and why.

(this is compounded by #91)

java.lang.ClassCastException: com.fnproject.fn.runtime.flow.SerUtils$2 cannot be cast to java.base/java.lang.String
com.fnproject.fn.api.flow.FlowCompletionException: java.lang.ClassCastException: com.fnproject.fn.runtime.flow.SerUtils$2 cannot be cast to java.base/java.lang.String
	at com.fnproject.fn.runtime.flow.RemoteCompleterApiClient.waitForCompletion(RemoteCompleterApiClient.java:245)
	at com.fnproject.fn.runtime.flow.RemoteCompleterApiClient.waitForCompletion(RemoteCompleterApiClient.java:264)
	at com.fnproject.fn.runtime.flow.RemoteFlow$RemoteFlowFuture.get(RemoteFlow.java:148)
	at com.fnproject.fn.integration.ExerciseEverything.handleRequest(ExerciseEverything.java:440)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.fnproject.fn.runtime.MethodFunctionInvoker.tryInvoke(MethodFunctionInvoker.java:41)
	at com.fnproject.fn.runtime.EntryPoint.run(EntryPoint.java:87)
	at com.fnproject.fn.runtime.EntryPoint.main(EntryPoint.java:27)
Caused by: java.lang.ClassCastException: com.fnproject.fn.runtime.flow.SerUtils$2 cannot be cast to java.base/java.lang.String
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.fnproject.fn.runtime.flow.FlowContinuationInvoker.invokeContinuation(FlowContinuationInvoker.java:184)
	at com.fnproject.fn.runtime.flow.FlowContinuationInvoker.lambda$tryInvoke$1(FlowContinuationInvoker.java:133)
	at com.fnproject.fn.runtime.ReadOnceInputEvent.consumeBody(ReadOnceInputEvent.java:56)
	at com.fnproject.fn.runtime.flow.FlowContinuationInvoker.tryInvoke(FlowContinuationInvoker.java:99)

In the documented usage of Java FDK, the api module is added by the user to their pom.xml. The runtime module is part of the base image. This should be fine initially, but after that the user is free to try to update either to a newer version.

Example 1: User changes to use a newer api module. This should be OK unless we have made a breaking change to api.

Example 2: User changes to use a new base image (including an updated runtime jar). The newer version of the runtime jar, may have been compiled against a newer version of api than is actually present. This may or may not be a problem depending on if any of the method/class signatures in api have changed or been added.

Example 1 shows that we can't make breaking changes to the api module classes/methods without risking breakage to our users, even if we use semver to signal a breaking change (because not all the code which calls api methods is under user control). Fine. Example 2 shows that we can't change methods in api which are called from runtime - even adding a new method could cause failures as that method will be compiled against but might not be present at runtime.

I'm hitting an issue with Fn Flow. The logs are a bit hard to parse and could easily be made a bit more readable.

time="2018-10-12T15:53:18Z" level=debug msg="An error occurred in function: com.fnproject.fn.api.flow.FunctionInvokeFailedException: Failed to call function\n" action="server.handleFnInvokeCall)-fm" app_id=01CJGZBEMKNG8G00GZJ0000003 call_id=01CSMFPSFSNG8G00GZJ000000V fnID=01CS1HZT5GNG8G00GZJ000000F fn_id=01CS1HZT5GNG8G00GZJ000000F id=01CSMFPSFSNG8G00GZJ000000V image="trip:0.0.227" user_log=true

-> fnID and fn_id are duplicates; call_id and id too.

'time="2018-10-12T15:53:18Z" level=debug msg="com.fnproject.fn.api.flow.FunctionInvokeFailedException: Failed to call function\n" action="server.handleFnInvokeCall)-fm" app_id=01CJGZBEMKNG8G00GZJ0000003 call_id=01CSMFPSFSNG8G00GZJ000000V fnID=01CS1HZT5GNG8G00GZJ000000F fn_id=01CS1HZT5GNG8G00GZJ000000F id=01CSMFPSFSNG8G00GZJ000000V image="trip:0.0.227" user_log=true'

-> in server.handleFnInvokeCall)-fm, ')-fm' looks strange to me.

And at the end, I still don't know why the function couldn't be called ;-)

Currently Jackson is built into the FDK - support making this optional