fasten-project / fasten Goto Github PK

View Code? Open in Web Editor NEW

85.0 85.0 28.0 137.83 MB

Analyse package dependency networks at the call graph level

Home Page: https://www.fasten-project.eu

License: Apache License 2.0

Java 99.39% Dockerfile 0.61%

dependencies package-management static-analysis

fasten's People

Contributors

Stargazers

Watchers

fasten's Issues

Improve RepoCloner

Is your feature request related to a problem? Please describe.

Plugin does not report any errors, even though they do happen.
Partners asked to copy commitTag and sourcesUrl from the output of POM Analyzer to the output of RepoCloner.

Describe the solution you'd like

Error handling should be improved
Plugin should produce commitTag and sourcesUrl along with repoPath and maven coordinate as an output

javagc-wala does not seems to take into account the -jre parameter

Command: java -jar javacg-wala-0.0.1-SNAPSHOT-with-dependencies.jar -jre /media/DATA/utils/jdk8 -c org.xwiki.commons:xwiki-commons-collection:12.4 -o .

The command work fine and produce an accurate JSON file but it also log the following warning:

WARNING: java_runtime_dir /Library/Java/JavaVirtualMachines/jdk1.8.0_241.jdk/Contents/Home in wala.properties is invalid. Using boot class path instead.

This seems to suggest that the -jre parameter was not really taken into account and that it instead tried some hardcoded test path.

Implement license and compliance plugin's reporter

The QMSTR plugin detects license and compliance information of Maven projects.
For D4.3 (compatibility solver), QMSTR needs an ad-hoc reporter that will ultimately integrate such information into the KnowledgeBase through a Kafka topic.

GraphMavenResolver reports packages without associated revisions

In the output of GraphMavenResolver.resolveFullDependencySet() for it.unimi.dsi / dsiutils / 2.2.2 (compile scope) we find dependencies like ${pom.groupId}:javax.servlet:1.0.0 containing what appear to be unresolved Maven variables. For what we could ascertain, these product have no associated revision in the database (i.e., they appear in the "packages" table but have no associated row in "package_versions"). They should not appear in the output, as the output is supposed to be formed by the revisions on which dsiutils-2.2.2 is dependent (and they probably should not appear in the database).

cos:cos:05Nov2002
org.jruby:jruby:1.0.1
javax.activation:activation:1.1
jline:jline:1.0
poi:poi:2.5.1
javax.xml:jsr173:1.0
commons-net:commons-net:2.2
jetty:org.mortbay.jetty:5.1.4
org.apache.tiles:tiles-jsp:2.0.6
javax.ejb:ejb:3.0
org.codehaus.cargo:cargo-core-container-jo:0.8
cglib:cglib:2.1_3
dom4j:dom4j:1.6.1
backport-util-concurrent:backport-util-concurrent:3.0
hibernate:hibernate:2.1.8
org.aspectj:aspectjweaver:1.6.1
org.apache.xmlgraphics:batik-dom:1.7
axion:axion:1.0-M3-dev
msv:relaxngDatatype:20030807
mondrian:mondrian:2.3.2.8944
commons-lang:commons-lang:2.6
javax.servlet:jstl:1.1.0
bsh:bsh:1.2b3
org.apache.lucene:lucene-core:2.0.0
commons-io:commons-io:2.4
com.ibm.icu:icu4j:2.6.1
activation:activation:1.0.2
org.codehaus.cargo:cargo-core-container-jetty:0.8
jfree:jfreechart:[1.0.0,)
org.apache.tiles:tiles-api:2.0.6
bsf:bsf:2.4.0
velocity-tools:velocity-tools-view:1.4
werken-xpath:werken-xpath:0.9.4
avalon-framework:avalon-framework:4.1.3
javax.jts:jts:1.0
org.slf4j:slf4j-jdk14:1.7.6
org.apache.xmlgraphics:batik-svggen:1.7
com.thoughtworks.xstream:xstream:1.4.2
ch.qos.logback:logback-classic:1.1.2
qdox:qdox:1.5
org.jboss.logging:jboss-logging-spi:2.1.2.GA
org.jboss.netty:netty:3.2.7.Final
org.springframework:spring-web:2.0.2
dbunit:dbunit:2.1
checkstyle:checkstyle-optional:4.3
com.ibatis:ibatis2:2.3.0.677
org.apache.xmlgraphics:batik-gvt:1.7
${pom.groupId}:javax.servlet:1.0.0
emma:emma_ant:2.1.5320
commons-collections:commons-collections-testframework:3.2.1
xdoclet:xjavadoc:1.1
junit-addons:junit-addons:1.4
ehcache:ehcache:1.2
org.codehaus.jettison:jettison:1.2
xmlpull:xmlpull:1.1.3.1
howl:howl-logger:0.1.11
msv:xsdlib:20030807
oro:oro:2.0.8
org.springframework:spring-webmvc:2.0.2
castor:castor:0.9.9.0-pre
commons-cli:commons-cli:1.0
postgresql:postgresql:8.4-701.jdbc4
asm:asm-attrs:2.2
org.apache.xmlgraphics:batik-script:1.7
com.caucho:hessian:3.1.3
radeox:radeox:0.9
org.slf4j:slf4j-simple:1.3.1
xerces:xercesImpl:2.4.0
org.jdom:jaxen-core:1.0-FCS
org.springframework:spring:2.5.6
stax:stax-api:1.0
org.aspectj:aspectjrt:1.6.1
org.jdom:saxpath:1.0-FCS
cglib:cglib-full:2.0.2
org.apache.geronimo.specs:geronimo-j2ee-connector_1.5_spec:1.0
aopalliance:aopalliance:1.0
jmock:jmock-cglib:1.2.0
org.apache.xmlgraphics:batik-awt-util:1.7
org.apache.mina:mina-filter-ssl:1.1.7
javax.mail:mail:1.4
net.sf.kxml:kxml2-min:2.3.0
xerces:xmlParserAPIs:2.6.2
org.objectweb.carol:carol:2.0.5
commons-beanutils:commons-beanutils-core:1.7.0
jboss:jboss-minimal:4.0.2
jstl:jstl:1.0.6
javax.transaction:jta:1.1
org.mockito:mockito-all:1.8.2
javax.xml:jaxrpc-api:1.1
jexcelapi:jxl:2.6.6
mx4j:mx4j:3.0.2
${pom.groupId}:org.osgi.core:1.4.0
taglibs:standard:1.1.2
poi:poi-2.0-final:20040126
org.slf4j:nlog4j:1.2.24
org.springframework:spring-aspects:2.0.2
commons-pool:commons-pool:1.4
regexp:regexp:1.3
org.codehaus.cargo:cargo-core-container-orion:0.8
org.apache.felix:org.osgi.core:1.4.0
org.apache.ant:ant-testutil:1.7.0
jboss:jboss-system:4.0.2
com.tonicsystems:jarjar:0.6
org.apache.geronimo.specs:geronimo-j2ee-deployment_1.1_spec:1.0
org.hibernate:hibernate-entitymanager:3.3.2.GA
org.springframework:spring-core:2.0.2
commons-beanutils:commons-beanutils:1.8.3
slide:webdavlib:2.0
javacc:javacc:3.2
org.apache.poi:poi:3.0.1-FINAL
org.fusesource.jansi:jansi:1.6
com.sun.jmx:jmxri:1.2.1
jdbm:jdbm:1.0
org.apache.felix:org.apache.felix.main:2.0.2
javax.resource:connector:1.0
cglib:cglib-nodep:2.2
jboss:jboss-j2se:200504122039
emma:emma:2.1.5320
org.testng:testng:6.5.2
org.bouncycastle:bcprov-jdk14:1.45
org.codehaus.janino:commons-compiler:2.6.1
org.apache.jackrabbit:jackrabbit-jcr-commons:1.5.2
javax.sql:jdbc-stdext:2.0
org.codehaus.cargo:cargo-core-api-util:0.8
commons-attributes:commons-attributes-api:2.2
opensymphony:oscache:2.1
org.apache.ant:ant-nodeps:1.7.1
org.apache.commons:commons-compress:1.1
javax.security:jacc:1.0
antlr:antlr:2.7.2
rhino:js:1.6R2
commons-jxpath:commons-jxpath:1.3
${pom.groupId}:org.apache.felix.shell:1.4.1
hibernate:antlr:2.7.5H3
org.springframework:spring-dao:2.0.2
aspectj:aspectjweaver:1.5.3
htmlunit:htmlunit:1.8
com.martiansoftware:jsap:2.1
com.icegreen:greenmail:1.3
tomcat:naming-java:5.0.28
httpunit:httpunit:1.6.1
org.codehaus.jcsp:jcsp:1.1-rc5
org.springframework:spring-aop:2.0.2
javax.xml.soap:saaj-api:1.3
bouncycastle:bcmail-jdk14:138
xom:xom:1.0b3
org.apache.felix:org.osgi.compendium:1.4.0
servletapi:servletapi:2.3
org.freemarker:freemarker:2.3.14
org.apache.xmlgraphics:batik-anim:1.7
groovy:groovy:1.0
org.codehaus.groovy:groovy:1.5.6
proxool:proxool:0.8.3
commons-primitives:commons-primitives:1.0
jboss:javassist:3.3.ga
com.caucho:burlap:2.1.12
ant:ant:1.5.1
org.codehaus.cargo:cargo-core-api-module:0.8
org.subethamail:subethasmtp:2.1.0
xpp3:xpp3:1.1.3.3
radeox:radeox-oro:0.9
opensymphony:quartz-all:1.6.0
org.apache.commons:commons-vfs2:2.0
commons-validator:commons-validator:1.3.1
org.springframework:spring-beans:2.0.2
org.codehaus.cargo:cargo-core-container-geronimo:0.8
jaxen:jaxen:1.1-beta-6
org.apache.bsf:bsf-api:3.1
org.codehaus.jsr166-mirror:jsr166y:1.7.0
commons-collections:commons-collections:20040616
commons-jelly:commons-jelly-tags-xml:1.0
commons-jelly:commons-jelly-tags-log:1.0
com.google.inject:guice:2.0
net.sourceforge.jexcelapi:jxl:2.6
org.slf4j:integration:1.7.6
org.codehaus.cargo:cargo-ant:0.8
com.mockrunner:mockrunner-jdk1.3-j2ee1.3:0.4
javax.resource:connector-api:1.5
xalan:xalan:2.5.1
org.codehaus.gpars:gpars:1.0.0
${pom.groupId}:org.apache.felix.shell.tui:1.4.1
org.codehaus.cargo:cargo-core-uberjar:0.8
jtidy:jtidy:4aug2000r7-dev
velocity:velocity-dep:1.4
org.jdom:jaxen-jdom:1.0-FCS
org.apache.mina:mina-integration-jmx:1.1.7
com.servlets:cos:05Nov2002
commons-configuration:commons-configuration:1.8
commons-modeler:commons-modeler:2.0
org.codehaus.cargo:cargo-core-api-generic:0.8
com.google.guava:guava:18.0
log4j:log4j:1.2.17
junitperf:junitperf:1.8
xerces:xerces-impl:2.6.2
org.apache.shale:shale-test:1.0.4
org.springframework:spring-jpa:2.0.2
org.apache.xmlgraphics:batik-bridge:1.7
geronimo-spec:geronimo-spec-jta:1.0.1B-rc2
com.jcraft:jsch:0.1.42
jasperreports:jasperreports:2.0.5
org.codehaus.cargo:cargo-core-container-tomcat:0.8
${pom.groupId}:org.apache.felix.bundlerepository:1.4.2
ch.qos.logback:logback-core:1.1.2
logkit:logkit:1.0.1
commons-jelly:commons-jelly-tags-junit:1.0
com.oracle:oc4j:1.0
stax:stax-ri:1.0
jotm:jotm_iiop_stubs:2.0.10
xmlunit:xmlunit:1.1
net.sf.kxml:kxml2:2.3.0
org.slf4j:jul-to-slf4j:1.7.6
org.apache.ibatis:ibatis-sqlmap:2.3.4.726
org.apache.geronimo.specs:geronimo-ejb_2.1_spec:1.0
org.hamcrest:hamcrest-core:1.3
org.apache.ant:ant-antlr:1.8.4
com.cenqua.clover:clover:1.3.13
org.apache.jackrabbit:jackrabbit-webdav:1.5.2
commons-attributes:commons-attributes-compiler:2.2
org.springframework:spring-remoting:1.2.8
javax.jms:jms:1.1
com.sun.jdmk:jmxtools:1.2.1
jgroups:jgroups-all:2.4.1
org.apache.xmlgraphics:batik-css:1.7
asm:asm-commons:2.2.3
com.bea.xml:jsr173-ri:1.0
org.apache.felix:org.osgi.foundation:1.2.0
c3p0:c3p0:0.9.1.2
jboss:jboss-archive-browsing:5.0.0alpha-200607201-119
woodstox:wstx-asl:3.2.2
org.apache.directory.server:apacheds-core:1.0-RC3
com.h2database:h2:1.2.132
jdom:jdom:1.0
com.lowagie:itext:2.0.7
tomcat:catalina:5.5.23
jmock:jmock:1.0.0
com.bea.wlplatform:commonj-twm:1.1
org.acegisecurity:acegi-security:1.0.3
stax:stax:1.2.0
org.apache.xmlgraphics:batik-xml:1.7
org.codehaus.cargo:cargo-core-container-jboss:0.8
com.google.code.findbugs:jsr305:1.3.9
groovy:groovy-all-minimal:1.0
org.apache.geronimo.specs:geronimo-servlet_2.4_spec:1.0
cactus:cactus:12-1.4.1
org.beanshell:bsh:2.0b4
org.codehaus.cargo:cargo-core-container-weblogic:0.8
joda-time:joda-time:1.6
org.hibernate:ejb3-persistence:1.0.1.GA
org.springframework:spring-context:2.0.2
commons-javaflow:commons-javaflow:20060411
org.codehaus.cargo:cargo-core-container-resin:0.8
struts:struts:1.2.9
org.slf4j:slf4j-ext:1.7.6
mockobjects:mockobjects-core:0.09
aspectj:aspectjrt:1.5.3
junit:junit:4.12-beta-1
xpp3:xpp3_min:1.1.4c
nekohtml:nekohtml:0.9.5
pull-parser:pull-parser:2
org.slf4j:slf4j-log4j12:1.7.6
com.thoughtworks.qdox:qdox:1.12
com.google.protobuf:protobuf-java:2.4.1
org.springframework:spring-support:1.2.8
org.mockito:mockito-core:1.9.0
org.apache.commons:commons-math3:3.3
com.oracle:toplink-essentials:2.41
org.codehaus.groovy:groovy-all:2.0.7
org.easymock:easymock:3.1
org.objenesis:objenesis:1.0
quartz:quartz:1.6.0
org.codehaus.cargo:cargo-core-api-container:0.8
xml-apis:xml-apis-ext:1.3.04
org.springframework:spring-jdbc:2.0.2
net.sf.hibernate:hibernate:2.1.8
saxpath:saxpath:1.0-FCS
javax.servlet:servlet-api:2.3
xml-resolver:xml-resolver:1.2
org.springframework:spring-agent:2.0.2
javax.jcr:jcr:1.0
commons-codec:commons-codec:1.5
org.codehaus.woodstox:wstx-asl:3.2.7
mockobjects:mockobjects-jdk1.4-j2ee1.3:0.09
org.apache.xmlgraphics:batik-util:1.7
javassist:javassist:3.4.GA
com.experlog:xapool:1.5.0
org.easymock:easymockclassextension:3.1
idb:idb:3.26
org.springframework:spring-mock:2.0.2
forehead:forehead:1.0-beta-5
commons-digester:commons-digester:1.8.1
xerces:xerces:2.4.0
com.oracle.toplink:toplink:10.1.3
org.apache.ant:ant-junit:1.8.4
javax.faces:jsf-api:1.1
org.jdom:jdom:1.1
jboss:jboss-jee:4.2.0.GA
xml-apis:xml-apis:1.0.b2
org.apache.geronimo.specs:geronimo-jms_1.1_spec:1.0
commons-dbcp:commons-dbcp:1.2.2
velocity:velocity:1.5
gsbase:gsbase:2.0.1
commons-jelly:commons-jelly:1.0
org.apache.commons:commons-jexl:2.1.1
openejb:openejb-loader:1.0
itext:itext:1.3
ch.qos.cal10n:cal10n-api:0.8.1
org.yaml:snakeyaml:1.6
asm:asm-tree:2.2.3
commons-logging:commons-logging:1.1.1
bouncycastle:bcprov-jdk14:138
org.apache.mina:mina-core:
jruby:jruby:0.9.2
commons-fileupload:commons-fileupload:1.2
jotm:jotm_jrmp_stubs:2.0.10
org.apache.xmlgraphics:batik-parser:1.7
org.apache.velocity:velocity:1.6.2
org.slf4j:log4j-over-slf4j:1.7.6
org.apache.openejb:javaee-api:5.0-2
org.ccil.cowan.tagsoup:tagsoup:0.9.7
swarmcache:swarmcache:1.0RC2
jaxme:jaxme-api:0.3
com.jamonapi:jamon:2.4
org.hibernate:hibernate-annotations:3.3.1.GA
${pom.groupId}:org.apache.felix.framework:2.0.2
jboss:jboss-cache:1.2.2
org.hibernate:hibernate:3.2.6.ga
org.json:json:20080701
org.apache.ant:ant-launcher:1.7.1
xstream:xstream:1.2
ant:ant-junit:1.6.5
org.bouncycastle:bcpg-jdk14:1.45
javax.portlet:portlet-api:1.0
it.unimi.dsi:fastutil:6.5.15
eclipse:jdtcore:[3.1.0,)
jfree:jcommon:[1.0.0,)
ch.qos.logback:logback-classic.jar:2.2.2
org.apache.geronimo.specs:geronimo-jta_1.0.1B_spec:1.0
ant:ant-nodeps:1.6.2
portlet-api:portlet-api:1.0
groovy:groovy-all:1.0-beta-10
openejb:openejb-core:1.0
org.mockejb:mockejb:0.6-beta2
commons-vfs:commons-vfs:1.0
mysql:mysql-connector-java:5.1.9
net.sf.ehcache:ehcache:1.5.0
javax.servlet:jsp-api:2.0
asm:asm-analysis:2.2
ant:ant-trax:1.6.2
com.keypoint:png-encoder:1.5
checkstyle:checkstyle:4.3
org.apache.ant:ant:1.7.1
javax.persistence:persistence-api:1.0
org.codehaus.janino:janino:2.6.1
commons-httpclient:commons-httpclient:2.0.2
mx4j:mx4j-jmx:2.1.1
javax.jdo:jdo2-api:2.0
com.beust:jcommander:1.12
commons-logging:commons-logging-api:1.1
org.apache.ivy:ivy:2.2.0
net.sf.jsr107cache:jsr107cache:1.0
commons-beanutils:commons-beanutils-bean-collections:1.7.0
org.slf4j:slf4j-api:1.7.7
commons-discovery:commons-discovery:20030211.213356
asm:asm-util:2.2.3
ant:ant-launcher:1.6.2
jotm:jotm:2.0.10
freemarker:freemarker:2.3.8
tomcat:naming-common:5.0.28
com.megginson.sax:xml-writer:0.2
org.hibernate:hibernate-commons-annotations:3.0.0.ga
org.multiverse:multiverse-beta:0.7-RC-1
com.jcraft:jzlib:1.0.7
org.apache.xmlgraphics:batik-ext:1.7
org.apache.directory.server:apacheds-core-shared:1.0-RC3
org.apache.tiles:tiles-core:2.0.6
concurrent:concurrent:1.3.4
asm:asm:2.2.3
org.apache.axis:axis:1.4
cglib-nodep:cglib-nodep:2.1_3
javax.j2ee:j2ee:1.4
org.apache.xmlgraphics:batik-svg-dom:1.7
hessian:hessian:3.0.20
jboss:jboss-common:4.0.2
velocity-tools:velocity-tools-generic:1.4
cactus:cactus-ant:1.4.1
org.apache.xmlgraphics:batik-js:1.7
hsqldb:hsqldb:1.8.0.7
commons-jexl:commons-jexl:1.0
cas:casclient:2.0.11

NullPointerException when generating the JSON string of the call graph of a specific Maven coordinate

The following Maven coordinate produces an unhandled NullPointerException:

ch.epfl.scala:collection-strawman_0.6:0.8.0

The stack trace for the issue is as follows:

java.lang.NullPointerException
	at eu.fasten.analyzer.javacgopal.data.callgraph.ExtendedRevisionCallGraph.toListOfString(ExtendedRevisionCallGraph.java:174)
	at eu.fasten.analyzer.javacgopal.data.callgraph.ExtendedRevisionCallGraph.lambda$toJSON$5(ExtendedRevisionCallGraph.java:158)
	at java.base/java.util.HashMap.forEach(HashMap.java:1336)
	at eu.fasten.analyzer.javacgopal.data.callgraph.ExtendedRevisionCallGraph.toJSON(ExtendedRevisionCallGraph.java:152)
	at eu.fasten.analyzer.javacgopal.Main.run(Main.java:123)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1729)
	at picocli.CommandLine.access$900(CommandLine.java:145)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2101)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2068)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:1935)
	at picocli.CommandLine.execute(CommandLine.java:1864)
	at eu.fasten.analyzer.javacgopal.Main.main(Main.java:136)

I temporarily patched the issue in 0b47f9e by adding a null pointer check in toListOfString method. The issue is that the FASTEN URIs of a type's methods are null and causes the described issue. I think that we should investigate this issue more and find out why a type has methods but its methods have not FASTEN URIs.

Reproduce the issue

1- Remove the null pointer check at

fasten/analyzer/javacg-opal/src/main/java/eu/fasten/analyzer/javacgopal/data/callgraph/ExtendedRevisionCallGraph.java

Line 174 in c9d73f1

if (fastenURI !=null) { result.add(fastenURI.toString()); }

2- Run the Main class of the OPAL plug-in with the given Maven coordinate.

RepoCloner tests failing

Describe the bug

Two Repo Cloner tests fail as the repository used in those tests no longer exists.
Filing tests:

HgClonerTest.cloneRepoTest()
HgClonerTest.cloneRepoWithVersionTest()

To Reproduce

Steps to reproduce the behavior:

Checkout develop branch
Run mvn install or manually run those tests

Expected behavior

Tests passing

Logs

No logs

Environment (please complete the following information):

OS: MacOS 11.00 Beta (Big Sur)
Java version: openjdk 11.0.7 2020-04-14

Additional context

No additional context

Add 'forge' field in the output topic of RepoCloner plugin

Problem

The code quality analyser needs the 'forge' information to decide the sub-process of code analysis.

Solution

Add 'forge' field in the output topic of RepoCloner plugin.

We need a tool which create a graph/tree taking a product as entry point and follow the calls in provided dependencies

Fasten Core provide a LocalMerger which takes as input an product graph and it's dependencies graphs to produce what its calling a merged graph but which in practice just complete the external calls located in the product's graph with the name of the product which actually contain the targeted call. Note that it's what is behind the javacg-opal --stitch CLI parameter documented as "Stitch artifact CG to dependencies" which I find a bit misleading.

In order to do most of the planned analysis (security, license, broken calls, etc.) we also need to follow each external call to produce a tree which start from the main product and produce a tree/graph of all the calls that may be directly or indirectly called through the main product.

We need a tool which provided with the same kind of parameters that LocalMerger gets (test1 graph and then a list of all its direct and transitive dependencies graphs) and produces a graph which contains test1 graph plus all the calls that could be found by recursively following the externals calls located in test1 graph.

Such a graph can then be be used as basis for various analysis.

Of course this is just a basic version and we might want to expose an API with a better memory footprint (we might have quite a lot of dependencies and associated graphs to parse).

Make sure the REST API handles concurrent requests

How does RESTeasy behave with multiple concurrent requests?

Few JOOQ queries (REST API) cause Jetty's idleTimeout

Describe the bug

This endpoints table includes the current status of every single endpoint.
The following endpoints use SQL queries that immediately return a result when executed from the PostgreSQL command line, but cause Jetty to timeout when executed with JOOQ:

/mvn/packages/{pkg}/versions
/mvn/packages/{pkg}/{pkg_ver}/callgraph
/mvn/packages/{pkg}/{pkg_ver}/callables
/mvn/packages/{pkg}/{pkg_ver}/{fasten_uri}/metadata
/mvn/packages/{pkg}/{pkg_ver}/edges

To Reproduce

Steps to reproduce the behavior:

Checkout to kbapi.

Compile the core and start the REST server:

mvn clean install -Dmaven.test.skip=true -f core/pom.xml && \
  PGPASSWORD=<KB_PWD> mvn clean install \
  -Dmaven.test.skip=true exec:java \
  -f analyzer/restapi-plugin/pom.xml \
  -Dexec.args='--port <REST_PORT> --database <KB_URL> --username <KB_USER>'

Perform a GET request to one of those endpoints, e.g.:

curl http://localhost:<REST_PORT>/mvn/packages/com.lemoulinstudio.jnag:jnag-utils-jse-rds/versions

Expected behavior

Those endpoints should return almost immediately.

Logs

Another endpoint working fine

[qtp1997591039-21] DEBUG org.jooq.tools.LoggerListener - Executing query          : select "public"."packages"."package_name", "public"."package_versions"."version", "public"."package_versions"."metadata" from "public"."packages" join "public"."package_versions" on "public"."packages"."id" = "public"."package_versions"."package_id" where (1 = 1 and lower("public"."packages"."package_name") = lower(?) and lower("public"."package_versions"."version") = lower(?))
[qtp1997591039-21] DEBUG org.jooq.tools.LoggerListener - -> with bind values      : select "public"."packages"."package_name", "public"."package_versions"."version", "public"."package_versions"."metadata" from "public"."packages" join "public"."package_versions" on "public"."packages"."id" = "public"."package_versions"."package_id" where (1 = 1 and lower("public"."packages"."package_name") = lower('com.lemoulinstudio.jnag:jnag-utils-jse-rds') and lower("public"."package_versions"."version") = lower('1.2.1'))
[qtp1997591039-21] DEBUG org.jooq.tools.LoggerListener - Fetched result           : +------------------------------------------+-------+--------------------------------------------------+
[qtp1997591039-21] DEBUG org.jooq.tools.LoggerListener -                          : |package_name                              |version|metadata                                          |
[qtp1997591039-21] DEBUG org.jooq.tools.LoggerListener -                          : +------------------------------------------+-------+--------------------------------------------------+
[qtp1997591039-21] DEBUG org.jooq.tools.LoggerListener -                          : |com.lemoulinstudio.jnag:jnag-utils-jse-rds|1.2.1  |{"commitTag": "", "sourcesUrl": "http://172.16....|
[qtp1997591039-21] DEBUG org.jooq.tools.LoggerListener -                          : +------------------------------------------+-------+--------------------------------------------------+
[qtp1997591039-21] DEBUG org.jooq.tools.LoggerListener - Fetched row(s)           : 1
[qtp1997591039-21] DEBUG eu.fasten.core.data.metadatadb.MetadataDao - Total rows: 1
[qtp1997591039-21] DEBUG org.jboss.resteasy.resteasy_jaxrs.i18n - MessageBodyWriter: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey
[qtp1997591039-21] DEBUG org.jboss.resteasy.resteasy_jaxrs.i18n - MessageBodyWriter: org.jboss.resteasy.plugins.providers.StringTextStar
[qtp1997591039-21] DEBUG org.jboss.resteasy.resteasy_jaxrs.i18n - MessageBodyWriter: org.jboss.resteasy.plugins.providers.StringTextStar
[qtp1997591039-21] DEBUG org.jboss.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.ServerWriterInterceptorContext,  Method : proceed
[qtp1997591039-21] DEBUG org.jboss.resteasy.resteasy_jaxrs.i18n - MessageBodyWriter: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey
[qtp1997591039-21] DEBUG org.jboss.resteasy.resteasy_jaxrs.i18n - MessageBodyWriter: org.jboss.resteasy.plugins.providers.StringTextStar
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.Server - handled=true async=false committed=false on HttpChannelOverHttp@4882f347{r=1,c=false,a=DISPATCHED,uri=//localhost:9090/mvn/packages/com.lemoulinstudio.jnag:jnag-utils-jse-rds/1.2.1/metadata,age=711}
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpChannelState - unhandle HttpChannelState@42a25c7a{s=DISPATCHED a=NOT_ASYNC i=true r=IDLE w=false}
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@4882f347{r=1,c=false,a=COMPLETING,uri=//localhost:9090/mvn/packages/com.lemoulinstudio.jnag:jnag-utils-jse-rds/1.2.1/metadata,age=712} action COMPLETE
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3db1efbc[p=0,l=644,c=32768,r=644]={<<<{"fields":[{"sche...ies\\": []}}"]]}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=true committing=true callback=Blocker@642e9155{null}
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpChannel - COMMIT for /mvn/packages/com.lemoulinstudio.jnag:jnag-utils-jse-rds/1.2.1/metadata on HttpChannelOverHttp@4882f347{r=1,c=true,a=COMPLETING,uri=//localhost:9090/mvn/packages/com.lemoulinstudio.jnag:jnag-utils-jse-rds/1.2.1/metadata,age=712}
200 null HTTP/1.1
Date: Thu, 15 Oct 2020 11:26:53 GMT
Content-Type: application/json


[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@38057c8a[PROCESSING][i=HTTP/1.1{s=200,h=2,cl=-1},cb=org.eclipse.jetty.server.HttpChannel$CommitCallback@13e215f0] generate: NEED_HEADER (null,[p=0,l=644,c=32768,r=644],true)@START
[qtp1997591039-21] DEBUG org.eclipse.jetty.http.HttpGenerator - generateHeaders HTTP/1.1{s=200,h=2,cl=-1} last=true content=DirectByteBuffer@3db1efbc[p=0,l=644,c=32768,r=644]={<<<{"fields":[{"sche...ies\\": []}}"]]}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}
[qtp1997591039-21] DEBUG org.eclipse.jetty.http.HttpGenerator - Date: Thu, 15 Oct 2020 11:26:53 GMT
Content-Type: application/json


[qtp1997591039-21] DEBUG org.eclipse.jetty.http.HttpGenerator - CONTENT_LENGTH
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@38057c8a[PROCESSING][i=HTTP/1.1{s=200,h=2,cl=-1},cb=org.eclipse.jetty.server.HttpChannel$CommitCallback@13e215f0] generate: FLUSH ([p=0,l=141,c=8192,r=141],[p=0,l=644,c=32768,r=644],true)@COMPLETING
[qtp1997591039-21] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@340e468f{IDLE}->null [HeapByteBuffer@1fa07425[p=0,l=141,c=8192,r=141]={<<<HTTP/1.1 200 OK\r\n....v20180320)\r\n\r\n>>>trol: max-age=0\r\n...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00},DirectByteBuffer@3db1efbc[p=0,l=644,c=32768,r=644]={<<<{"fields":[{"sche...ies\\": []}}"]]}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
[qtp1997591039-21] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@340e468f{WRITING}->null:IDLE-->WRITING
[qtp1997591039-21] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 785 SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=W,to=743/30000}{io=0/0,kio=0,kro=1}->HttpConnection@35760168[p=HttpParser{s=END,0 of -1},g=HttpGenerator@60d0c2e4{s=COMPLETING}]=>HttpChannelOverHttp@4882f347{r=1,c=true,a=COMPLETING,uri=//localhost:9090/mvn/packages/com.lemoulinstudio.jnag:jnag-utils-jse-rds/1.2.1/metadata,age=734}
[qtp1997591039-21] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=785 remaining=0 WriteFlusher@340e468f{WRITING}->null
[qtp1997591039-21] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@340e468f{IDLE}->null:WRITING-->IDLE
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@38057c8a[PROCESSING][i=HTTP/1.1{s=200,h=2,cl=-1},cb=org.eclipse.jetty.server.HttpChannel$CommitCallback@13e215f0] generate: DONE ([p=141,l=141,c=8192,r=0],[p=644,l=644,c=32768,r=0],true)@END
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpChannelState - onComplete HttpChannelState@42a25c7a{s=COMPLETING a=NOT_ASYNC i=false r=IDLE w=false}
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpChannel - COMPLETE for /mvn/packages/com.lemoulinstudio.jnag:jnag-utils-jse-rds/1.2.1/metadata written=644
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpChannelState - recycle HttpChannelState@42a25c7a{s=COMPLETED a=NOT_ASYNC i=false r=IDLE w=false}
[qtp1997591039-21] DEBUG org.eclipse.jetty.http.HttpParser - reset HttpParser{s=END,0 of -1}
[qtp1997591039-21] DEBUG org.eclipse.jetty.http.HttpParser - END --> START
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0} handle exit, result COMPLETE
[qtp1997591039-21] DEBUG org.eclipse.jetty.io.ChannelEndPoint - filled 0 SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=7/30000}{io=0/0,kio=0,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}
[qtp1997591039-21] DEBUG org.eclipse.jetty.io.ChannelEndPoint - filled 0 SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=7/30000}{io=0/0,kio=0,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpConnection - HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}<-SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=8/30000}{io=0/0,kio=0,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0} filled 0 HeapByteBuffer@1fa07425[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 200 OK\r\n...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpConnection - HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}<-SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=8/30000}{io=0/0,kio=0,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0} parse HeapByteBuffer@1fa07425[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 200 OK\r\n...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} {}
[qtp1997591039-21] DEBUG org.eclipse.jetty.http.HttpParser - parseNext s=START HeapByteBuffer@1fa07425[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 200 OK\r\n...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpConnection - HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}<-SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=8/30000}{io=0/0,kio=0,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0} parsed false HttpParser{s=START,0 of -1}
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpConnection - releaseRequestBuffer HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}<-SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=9/30000}{io=0/0,kio=0,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}
[qtp1997591039-21] DEBUG org.eclipse.jetty.io.AbstractConnection - fillInterested HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}<-SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=9/30000}{io=0/0,kio=0,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}
[qtp1997591039-21] DEBUG org.eclipse.jetty.io.FillInterest - interested FillInterest@26c69379{AC.ReadCB@35760168{HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}<-SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=FI,flush=-,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}}}
[qtp1997591039-21] DEBUG org.eclipse.jetty.io.ChannelEndPoint - changeInterests p=false 0->1 for SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=FI,flush=-,to=0/30000}{io=0/1,kio=0,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}
[qtp1997591039-21] DEBUG org.eclipse.jetty.io.ManagedSelector - Queued change org.eclipse.jetty.io.ChannelEndPoint$1@4067b2a7 on org.eclipse.jetty.io.ManagedSelector@73afe5e id=1 keys=1 selected=0 updates=0
[qtp1997591039-19] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector sun.nio.ch.KQueueSelectorImpl@19a651e1 woken up from select, 0/1 selected
[qtp1997591039-19] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector sun.nio.ch.KQueueSelectorImpl@19a651e1 processing 0 keys, 1 updates
[qtp1997591039-19] DEBUG org.eclipse.jetty.io.ManagedSelector - updateable 1
[qtp1997591039-19] DEBUG org.eclipse.jetty.io.ManagedSelector - update org.eclipse.jetty.io.ChannelEndPoint$1@4067b2a7
[qtp1997591039-19] DEBUG org.eclipse.jetty.io.ChannelEndPoint - Key interests updated 0 -> 1 on SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=FI,flush=-,to=1/30000}{io=1/1,kio=1,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}
[qtp1997591039-19] DEBUG org.eclipse.jetty.io.ManagedSelector - updates 0
[qtp1997591039-19] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector sun.nio.ch.KQueueSelectorImpl@19a651e1 waiting on select
[qtp1997591039-21] DEBUG org.eclipse.jetty.server.HttpConnection - HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}<-SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=FI,flush=-,to=1/30000}{io=1/1,kio=0,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0} onFillable exit HttpChannelState@42a25c7a{s=IDLE a=NOT_ASYNC i=true r=IDLE w=false} null
[qtp1997591039-21] DEBUG org.eclipse.jetty.util.thread.QueuedThreadPool - ran CEP:SocketChannelEndPoint@6ea8184{/0:0:0:0:0:0:0:1:51647<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=FI,flush=-,to=2/30000}{io=1/1,kio=1,kro=1}->HttpConnection@35760168[p=HttpParser{s=START,0 of -1},g=HttpGenerator@60d0c2e4{s=START}]=>HttpChannelOverHttp@4882f347{r=1,c=false,a=IDLE,uri=null,age=0}:runFillable:BLOCKING

Jetty timeout

12:47:41.586 [qtp1703173670-21] DEBUG org.jooq.tools.LoggerListener - Executing query          : select "public"."edges"."source_id", "public"."edges"."target_id", "public"."edges"."receivers", "public"."edges"."metadata" from "public"."edges" join "public"."callables" on "public"."edges"."source_id" = "public"."callables"."id" join "public"."modules" on "public"."modules"."id" = "public"."callables"."module_id" join "public"."package_versions" on "public"."package_versions"."id" = "public"."modules"."package_version_id" join "public"."packages" on "public"."packages"."id" = "public"."package_versions"."package_id" where (1 = 1 and lower("public"."packages"."package_name") = lower(?) and lower("public"."package_versions"."version") = lower(?))
12:47:41.588 [qtp1703173670-21] DEBUG org.jooq.tools.LoggerListener - -> with bind values      : select "public"."edges"."source_id", "public"."edges"."target_id", "public"."edges"."receivers", "public"."edges"."metadata" from "public"."edges" join "public"."callables" on "public"."edges"."source_id" = "public"."callables"."id" join "public"."modules" on "public"."modules"."id" = "public"."callables"."module_id" join "public"."package_versions" on "public"."package_versions"."id" = "public"."modules"."package_version_id" join "public"."packages" on "public"."packages"."id" = "public"."package_versions"."package_id" where (1 = 1 and lower("public"."packages"."package_name") = lower('com.lemoulinstudio.jnag:jnag-utils-jse-rds') and lower("public"."package_versions"."version") = lower('1.2.1'))
12:48:11.158 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.IdleTimeout - SocketChannelEndPoint@40c34170{/0:0:0:0:0:0:0:1:51318<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=29946/30000}{io=0/0,kio=0,kro=1}->HttpConnection@767f9a16[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7fd061ea{s=START}]=>HttpChannelOverHttp@6f71a71e{r=1,c=false,a=DISPATCHED,uri=//localhost:9090/mvn/packages/com.lemoulinstudio.jnag:jnag-utils-jse-rds/1.2.1/edges,age=29935} idle timeout check, elapsed: 29946 ms, remaining: 54 ms
12:48:11.159 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.IdleTimeout - SocketChannelEndPoint@5b57f627{/0:0:0:0:0:0:0:1:51319<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=FI,flush=-,to=29957/30000}{io=1/1,kio=1,kro=0}->HttpConnection@2c52309f[p=HttpParser{s=START,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0} idle timeout check, elapsed: 29957 ms, remaining: 43 ms
12:48:11.207 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.IdleTimeout - SocketChannelEndPoint@5b57f627{/0:0:0:0:0:0:0:1:51319<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=FI,flush=-,to=30005/30000}{io=1/1,kio=1,kro=0}->HttpConnection@2c52309f[p=HttpParser{s=START,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0} idle timeout check, elapsed: 30005 ms, remaining: -5 ms
12:48:11.207 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.IdleTimeout - SocketChannelEndPoint@5b57f627{/0:0:0:0:0:0:0:1:51319<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=FI,flush=-,to=30005/30000}{io=1/1,kio=1,kro=0}->HttpConnection@2c52309f[p=HttpParser{s=START,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0} idle timeout expired
12:48:11.211 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.FillInterest - onFail FillInterest@7673e53c{AC.ReadCB@2c52309f{HttpConnection@2c52309f[p=HttpParser{s=START,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0}<-SocketChannelEndPoint@5b57f627{/0:0:0:0:0:0:0:1:51319<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=FI,flush=-,to=30006/30000}{io=1/1,kio=1,kro=0}->HttpConnection@2c52309f[p=HttpParser{s=START,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0}}}
java.util.concurrent.TimeoutException: Idle timeout expired: 30005/30000 ms
	at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166)
	at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
12:48:11.211 [Scheduler-20081582] DEBUG org.eclipse.jetty.http.HttpParser - close HttpParser{s=START,0 of -1}
12:48:11.211 [Scheduler-20081582] DEBUG org.eclipse.jetty.http.HttpParser - START --> CLOSE
12:48:11.212 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.AbstractConnection - HttpConnection@2c52309f[p=HttpParser{s=CLOSE,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0}<-SocketChannelEndPoint@5b57f627{/0:0:0:0:0:0:0:1:51319<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=30010/30000}{io=1/1,kio=1,kro=0}->HttpConnection@2c52309f[p=HttpParser{s=CLOSE,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0} onFillInterestedFailed {}
12:48:11.212 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.AbstractEndPoint - shutdownOutput SocketChannelEndPoint@5b57f627{/0:0:0:0:0:0:0:1:51319<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=30010/30000}{io=1/1,kio=1,kro=0}->HttpConnection@2c52309f[p=HttpParser{s=CLOSE,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0}
12:48:11.213 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.AbstractConnection - fillInterested HttpConnection@2c52309f[p=HttpParser{s=CLOSE,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0}<-SocketChannelEndPoint@5b57f627{/0:0:0:0:0:0:0:1:51319<->/0:0:0:0:0:0:0:1:9090,OSHUT,fill=-,flush=-,to=30011/30000}{io=1/1,kio=1,kro=0}->HttpConnection@2c52309f[p=HttpParser{s=CLOSE,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0}
12:48:11.229 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.FillInterest - interested FillInterest@7673e53c{AC.ReadCB@2c52309f{HttpConnection@2c52309f[p=HttpParser{s=CLOSE,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0}<-SocketChannelEndPoint@5b57f627{/0:0:0:0:0:0:0:1:51319<->/0:0:0:0:0:0:0:1:9090,OSHUT,fill=FI,flush=-,to=1/30000}{io=1/1,kio=1,kro=0}->HttpConnection@2c52309f[p=HttpParser{s=CLOSE,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0}}}
12:48:11.229 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.ChannelEndPoint - changeInterests p=false 1->1 for SocketChannelEndPoint@5b57f627{/0:0:0:0:0:0:0:1:51319<->/0:0:0:0:0:0:0:1:9090,OSHUT,fill=FI,flush=-,to=1/30000}{io=1/1,kio=1,kro=0}->HttpConnection@2c52309f[p=HttpParser{s=CLOSE,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0}
12:48:11.229 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.ManagedSelector - Queued change org.eclipse.jetty.io.ChannelEndPoint$1@1f40512 on org.eclipse.jetty.io.ManagedSelector@795c2808 id=2 keys=1 selected=0 updates=0
12:48:11.229 [qtp1703173670-18] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector sun.nio.ch.KQueueSelectorImpl@2a08db6b woken up from select, 0/1 selected
12:48:11.229 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.WriteFlusher - ignored: WriteFlusher@289da003{IDLE}->null
java.util.concurrent.TimeoutException: Idle timeout expired: 30005/30000 ms
	at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166)
	at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
12:48:11.229 [qtp1703173670-18] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector sun.nio.ch.KQueueSelectorImpl@2a08db6b processing 0 keys, 1 updates
12:48:11.229 [qtp1703173670-18] DEBUG org.eclipse.jetty.io.ManagedSelector - updateable 1
12:48:11.229 [qtp1703173670-18] DEBUG org.eclipse.jetty.io.ManagedSelector - update org.eclipse.jetty.io.ChannelEndPoint$1@1f40512
12:48:11.229 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.AbstractEndPoint - Ignored idle endpoint SocketChannelEndPoint@5b57f627{/0:0:0:0:0:0:0:1:51319<->/0:0:0:0:0:0:0:1:9090,OSHUT,fill=FI,flush=-,to=1/30000}{io=1/1,kio=1,kro=0}->HttpConnection@2c52309f[p=HttpParser{s=CLOSE,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0}
12:48:11.230 [qtp1703173670-18] DEBUG org.eclipse.jetty.io.ChannelEndPoint - Key interests updated 1 -> 1 on SocketChannelEndPoint@5b57f627{/0:0:0:0:0:0:0:1:51319<->/0:0:0:0:0:0:0:1:9090,OSHUT,fill=FI,flush=-,to=1/30000}{io=1/1,kio=1,kro=0}->HttpConnection@2c52309f[p=HttpParser{s=CLOSE,0 of -1},g=HttpGenerator@6883fe6e{s=START}]=>HttpChannelOverHttp@43a0f6a5{r=0,c=false,a=IDLE,uri=null,age=0}
12:48:11.230 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.IdleTimeout - SocketChannelEndPoint@40c34170{/0:0:0:0:0:0:0:1:51318<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=30018/30000}{io=0/0,kio=0,kro=1}->HttpConnection@767f9a16[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7fd061ea{s=START}]=>HttpChannelOverHttp@6f71a71e{r=1,c=false,a=DISPATCHED,uri=//localhost:9090/mvn/packages/com.lemoulinstudio.jnag:jnag-utils-jse-rds/1.2.1/edges,age=30007} idle timeout check, elapsed: 30017 ms, remaining: -17 ms
12:48:11.230 [qtp1703173670-18] DEBUG org.eclipse.jetty.io.ManagedSelector - updates 0
12:48:11.230 [qtp1703173670-18] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector sun.nio.ch.KQueueSelectorImpl@2a08db6b waiting on select
12:48:11.230 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.IdleTimeout - SocketChannelEndPoint@40c34170{/0:0:0:0:0:0:0:1:51318<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=30018/30000}{io=0/0,kio=0,kro=1}->HttpConnection@767f9a16[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7fd061ea{s=START}]=>HttpChannelOverHttp@6f71a71e{r=1,c=false,a=DISPATCHED,uri=//localhost:9090/mvn/packages/com.lemoulinstudio.jnag:jnag-utils-jse-rds/1.2.1/edges,age=30007} idle timeout expired
12:48:11.230 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.FillInterest - onFail FillInterest@44511e5a{null}
java.util.concurrent.TimeoutException: Idle timeout expired: 30017/30000 ms
	at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166)
	at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
12:48:11.230 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.WriteFlusher - ignored: WriteFlusher@6cc0c546{IDLE}->null
java.util.concurrent.TimeoutException: Idle timeout expired: 30017/30000 ms
	at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166)
	at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
12:48:11.230 [Scheduler-20081582] DEBUG org.eclipse.jetty.io.AbstractEndPoint - Ignored idle endpoint SocketChannelEndPoint@40c34170{/0:0:0:0:0:0:0:1:51318<->/0:0:0:0:0:0:0:1:9090,OPEN,fill=-,flush=-,to=30018/30000}{io=0/0,kio=0,kro=1}->HttpConnection@767f9a16[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7fd061ea{s=START}]=>HttpChannelOverHttp@6f71a71e{r=1,c=false,a=DISPATCHED,uri=//localhost:9090/mvn/packages/com.lemoulinstudio.jnag:jnag-utils-jse-rds/1.2.1/edges,age=30007}

Environment

OS: macOS 10.15.6 (19G2021)
Java version: 11.0.2

Retrieving edges from KB takes too long

Is your feature request related to a problem? Please describe.

#80 aims to implement the REST API to access the KnowledgeBase.
The edges endpoint GET /mvn/packages/{pkg}/{pkg_ver}/edges should return all incoming and outgoing edges of a given package version.

The only way to access those edges is through the callables table, as visible from the schema.
For this reason, I am creating a view containing all callables given a package name and version:

CREATE OR REPLACE VIEW callable_ids AS
SELECT  c.id
FROM    packages AS p JOIN
        package_versions AS pv ON p.id = pv.package_id JOIN
        modules AS m ON pv.id = m.package_version_id JOIN
        callables AS c ON m.id = c.module_id
WHERE   p.package_name = <packageName> AND
        pv.version = <packageVersion>

A quick sketch of its algebraic tree shows that it might theoretically contain up to ~1.9B entries, in the absolute worst case ever (see details below), so this could theoretically be a first problem. However, for the sake of this issue, I will be using a package name and version that will cause this view to contain exactly 30 rows, i.e., package com.lemoulinstudio.jnag:jnag-utils-jse-rds, version 1.2.1.

Quick algebraic tree reasoning

The packages table

It contains ~200K rows. Its WHERE clause narrows its contribution to 1, given the fact that:
```
SELECT COUNT(DISTINCT package_name) FROM packages
```
returns exactly the same number of rows it contains.
The package_versions table

It has ~2.8M entries, with a reduction factor of ~100K for its version field, so on average ~28 possible entries for a given version.
Inner join between packages and package_versions

In the worst case, there might be all package versions for the selected package, resulting in a contribution of ~28 rows.
The modules table

~22.3M rows.
Inner join between 3. and modules

In the worst case, all modules may refer to an existing package (version), resulting in a contribution of ~22.3M rows.
The callables table

~1.9B rows.
Inner join between 5. and callables

In the worst case, all callables belong to all modules and package versions, resulting in a contribution of ~1.9B rows.

Such a view would be used by the following query:

SELECT  *
FROM    edges
WHERE   source_id IN (SELECT id FROM callable_ids) OR
        target_id IN (SELECT id FROM callable_ids)

The edges table is 3473 GB:

SELECT  pg_size_pretty (pg_relation_size('edges'));

With roughly 22 billion rows:

SELECT  n_live_tup
FROM    pg_stat_user_tables
WHERE   relname = 'edges';

This takes way too long, causing our Servlet engine to timeout.

Describe the solution you'd like

Personally, I would avoid using the edges table completely and only use the graph database which I have heard mentioned (@ilyagrishkov any news on this? 🙂).
If I'm not wrong, the only argument in favor of maintaining such a table was the lack of the source_id and target_id fields in the graph database.
_{On a side note, using that graph database might also help with the callables endpoints.}

To conclude, while I acknowledge that it may be hard to enforce consistency in two different databases, I do not know whether it would be worth to create indexes on those two fields.
Given its size, it is impossible to determine how many different values of callable IDs the edges table contains, i.e.:

SELECT  COUNT(DISTINCT source_id)
FROM    edges

If this number is high, Postgres might perform a table scan anyway.

Another option would be to insert redundancy so that I can avoid traversing big portions of the KnowledgeBase's E-R diagram while querying (in particular, packages.package_name and package_versions.version in callables, if possible/makes sense).

License compatibility of dependencies

Hello, I see that Fasten has a dependency on the WebGraph library. Fasten is Apache License (ASL) while WebGraph is GPL and this may be an issue:

Ref in the POM: https://github.com/fasten-project/fasten/blob/master/core/pom.xml#L40

As per ASL:

Apache 2 software can therefore be included in GPLv3 projects, because the GPLv3 license accepts our software into GPLv3 works. However, GPLv3 software cannot be included in Apache projects. The licenses are incompatible in one direction only, and it is a result of ASF's licensing philosophy and the GPLv3 authors' interpretation of copyright law.
...
We avoid GPLv3 software because merely linking to it is considered by the GPLv3 authors to create a derivative work. We want to honor their license. Unless GPLv3 licensors relax this interpretation of their own license regarding linking, our licensing philosophies are fundamentally incompatible. This is an identical issue for both GPLv2 and GPLv3.

You might want to have this verified.

Regards.

Release the project on Maven

Is your feature request related to a problem? Please describe.

It's currently impossible to have dependencies on some modules of the project from outside the project. E.g. WALA and OPAL v1 call graph generators are in different repositories and both of them need a dependency on the FASTEN core. Having FASTEN released on Maven would allow to have those dependencies.

Describe the solution you'd like

Creating a workflow that would push the project to Maven on every release would solve this problem.

Additional context

Having Maven releases will also increase maintainability of plugins in different repositories that depend on FASTEN as the code will be consistent across dependent project.

Change callables' metadata endpoint (and query)

Problem

As pointed out by @zikanl is his last email, one of the reasons why the query behind the callables' metadata endpoint (/mvn/packages/{pkg}/{pkg_ver}/{fasten_uri}/metadata) causes Jetty to timeout could be a complete scan of the callables table, as we're not specifying the module ID as a path parameter (see the Knowledge Base schema).

Also, according to the Wiki (and to my last call with @proksch & @ilyagrishkov):

Callable ID = module ID (package version) + FASTEN URI + other

May I ask for more info about that?

Proposed solution

I'd change /mvn/packages/{pkg}/{pkg_ver}/{fasten_uri}/metadata to one of the following:

/mvn/packages/{pkg}/{pkg_ver}/{module}/{callable}/metadata
/mvn/packages/{pkg}/{pkg_ver}/{module}/{callable}/metadata?fasten_uri=...
/mvn/packages/{pkg}/{pkg_ver}/{module}/{callable}/{fasten_uri}/metadata
/mvn/packages/{pkg}/{pkg_ver}/{module}/{fasten_uri}/metadata

Consequently, the corresponding query would get a new WHERE clause.

Getting org.dom4j.InvalidXPathException when trying to generate org.slf4j:slf4j-api:1.7.29 call graph

version: #4ee04b4321830d41d61dc93ab2e290026469411f (May 22, 2020)

Built with Java 11.0.7 with mvn install.

I execute the command given as example in the README file and get the following result:

$ java -jar docker/plugins/javacg-wala-0.0.1-SNAPSHOT-with-dependencies.jar -jre /media/DATA/utils/jdk8 -c org.slf4j:slf4j-api:1.7.29 --stdout
[main] INFO eu.fasten.analyzer.javacgwala.Main - Could not load wala.properties
[main] INFO eu.fasten.analyzer.javacgwala.data.callgraph.CallGraphConstructor - Generating call graph for the Maven coordinate using WALA: org.slf4j:slf4j-api:1.7.29
WARNING: java_runtime_dir /Library/Java/JavaVirtualMachines/jdk1.8.0_241.jdk/Contents/Home in wala.properties is invalid.  Using boot class path instead.
[main] INFO eu.fasten.analyzer.javacgwala.data.callgraph.CallGraphConstructor - Generated the call graph in 4.379 seconds.
[main] INFO eu.fasten.analyzer.javacgwala.data.callgraph.analyzer.WalaResultAnalyzer - Wrapping call graph with 4848 nodes...
[main] INFO eu.fasten.analyzer.javacgwala.data.callgraph.analyzer.WalaResultAnalyzer - Wrapped call graph in 0.497 seconds [Internal calls: 345, External calls: 219]
[main] ERROR eu.fasten.analyzer.javacgwala.Main - Failed to generate a call graph for Maven coordinate: org.slf4j:slf4j-api:1.7.29, Error: {}
org.dom4j.InvalidXPathException: Invalid XPath expression: ./*[local-name() ='properties'] org.jaxen.saxpath.base.XPathReader
	at org.dom4j.xpath.DefaultXPath.parse(DefaultXPath.java:355)
	at org.dom4j.xpath.DefaultXPath.<init>(DefaultXPath.java:59)
	at org.dom4j.DocumentFactory.createXPath(DocumentFactory.java:222)
	at org.dom4j.tree.AbstractNode.createXPath(AbstractNode.java:202)
	at org.dom4j.tree.AbstractNode.selectSingleNode(AbstractNode.java:178)
	at eu.fasten.analyzer.baseanalyzer.MavenCoordinate$MavenResolver.getDependencies(MavenCoordinate.java:182)
	at eu.fasten.analyzer.baseanalyzer.MavenCoordinate$MavenResolver.resolveDependencies(MavenCoordinate.java:160)
	at eu.fasten.analyzer.javacgwala.data.callgraph.PartialCallGraph.createExtendedRevisionCallGraph(PartialCallGraph.java:128)
	at eu.fasten.analyzer.javacgwala.Main.run(Main.java:179)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1729)
	at picocli.CommandLine.access$900(CommandLine.java:145)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2101)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2068)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:1935)
	at picocli.CommandLine.execute(CommandLine.java:1864)
	at eu.fasten.analyzer.javacgwala.Main.main(Main.java:133)

Maybe mvn install is not enough and it needs some other parameter ?

Here is the build log

[INFO] Scanning for projects...
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for eu.fasten.analyzer:javacg-wala:jar:0.0.1-SNAPSHOT
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found duplicate declaration of plugin org.apache.maven.plugins:maven-compiler-plugin @ eu.fasten:parent:0.0.1-SNAPSHOT, /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/pom.xml, line 86, column 21
[WARNING] 
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING] 
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING] 
[INFO] 
[INFO] -------------------< eu.fasten.analyzer:javacg-wala >-------------------
[INFO] Building javacg-wala 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ javacg-wala ---
[INFO] Deleting /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/target
[INFO] 
[INFO] --- jacoco-maven-plugin:0.8.2:prepare-agent (default) @ javacg-wala ---
[INFO] argLine set to -javaagent:/home/tmortagne/.m2/repository/org/jacoco/org.jacoco.agent/0.8.2/org.jacoco.agent-0.8.2-runtime.jar=destfile=/media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/target/jacoco.exec
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ javacg-wala ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ javacg-wala ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 14 source files to /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ javacg-wala ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 7 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ javacg-wala ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 5 source files to /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/target/test-classes
[INFO] /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/src/test/java/eu/fasten/analyzer/javacgwala/WALAPluginTest.java: /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/src/test/java/eu/fasten/analyzer/javacgwala/WALAPluginTest.java uses unchecked or unsafe operations.
[INFO] /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/src/test/java/eu/fasten/analyzer/javacgwala/WALAPluginTest.java: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ javacg-wala ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running eu.fasten.analyzer.javacgwala.WALAPluginTest
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.003 s - in eu.fasten.analyzer.javacgwala.WALAPluginTest
[INFO] Running eu.fasten.analyzer.javacgwala.data.ArtifactResolverTest
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 s - in eu.fasten.analyzer.javacgwala.data.ArtifactResolverTest
[INFO] Running eu.fasten.analyzer.javacgwala.data.core.MethodTest
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.001 s - in eu.fasten.analyzer.javacgwala.data.core.MethodTest
[INFO] Running eu.fasten.analyzer.javacgwala.data.callgraph.WalaResultAnalyzerTest
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 s - in eu.fasten.analyzer.javacgwala.data.callgraph.WalaResultAnalyzerTest
[INFO] Running eu.fasten.analyzer.javacgwala.data.callgraph.PartialCallGraphTest
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 s - in eu.fasten.analyzer.javacgwala.data.callgraph.PartialCallGraphTest
[INFO] 
[INFO] Results:
[INFO] 
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 5
[INFO] 
[INFO] 
[INFO] --- jacoco-maven-plugin:0.8.2:report (report) @ javacg-wala ---
[INFO] Loading execution data file /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/target/jacoco.exec
[INFO] Analyzed bundle 'javacg-wala' with 18 classes
[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ javacg-wala ---
[INFO] Building jar: /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/target/javacg-wala-0.0.1-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-shade-plugin:3.2.1:shade (default) @ javacg-wala ---
[INFO] Including eu.fasten:core:jar:0.0.1-SNAPSHOT in the shaded jar.
[INFO] Including it.unimi.dsi:dsiutils:jar:2.6.0 in the shaded jar.
[INFO] Including it.unimi.dsi:fastutil:jar:8.2.3 in the shaded jar.
[INFO] Including com.martiansoftware:jsap:jar:2.1 in the shaded jar.
[INFO] Including com.google.guava:guava:jar:28.0-jre in the shaded jar.
[INFO] Including com.google.guava:failureaccess:jar:1.0.1 in the shaded jar.
[INFO] Including com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava in the shaded jar.
[INFO] Including com.google.code.findbugs:jsr305:jar:3.0.2 in the shaded jar.
[INFO] Including org.checkerframework:checker-qual:jar:2.8.1 in the shaded jar.
[INFO] Including com.google.errorprone:error_prone_annotations:jar:2.3.2 in the shaded jar.
[INFO] Including com.google.j2objc:j2objc-annotations:jar:1.3 in the shaded jar.
[INFO] Including org.codehaus.mojo:animal-sniffer-annotations:jar:1.17 in the shaded jar.
[INFO] Including commons-configuration:commons-configuration:jar:1.10 in the shaded jar.
[INFO] Including commons-lang:commons-lang:jar:2.6 in the shaded jar.
[INFO] Including commons-logging:commons-logging:jar:1.1.1 in the shaded jar.
[INFO] Including commons-io:commons-io:jar:2.6 in the shaded jar.
[INFO] Including commons-collections:commons-collections:jar:20040616 in the shaded jar.
[INFO] Including org.apache.commons:commons-math3:jar:3.6.1 in the shaded jar.
[INFO] Including it.unimi.dsi:webgraph:jar:3.6.3 in the shaded jar.
[INFO] Including it.unimi.dsi:sux4j:jar:5.0.4 in the shaded jar.
[INFO] Including net.sf.jung:jung-api:jar:2.1.1 in the shaded jar.
[INFO] Including net.sf.jung:jung-io:jar:2.1.1 in the shaded jar.
[INFO] Including net.sf.jung:jung-algorithms:jar:2.1.1 in the shaded jar.
[INFO] Including org.apache.commons:commons-lang3:jar:3.9 in the shaded jar.
[INFO] Including org.eclipse.jgit:org.eclipse.jgit:jar:5.2.1.201812262042-r in the shaded jar.
[INFO] Including com.jcraft:jsch:jar:0.1.54 in the shaded jar.
[INFO] Including com.jcraft:jzlib:jar:1.1.1 in the shaded jar.
[INFO] Including com.googlecode.javaewah:JavaEWAH:jar:1.1.6 in the shaded jar.
[INFO] Including org.json:json:jar:20180813 in the shaded jar.
[INFO] Including org.apache.kafka:kafka-clients:jar:2.3.0 in the shaded jar.
[INFO] Including com.github.luben:zstd-jni:jar:1.4.0-1 in the shaded jar.
[INFO] Including org.lz4:lz4-java:jar:1.6.0 in the shaded jar.
[INFO] Including org.xerial.snappy:snappy-java:jar:1.1.7.3 in the shaded jar.
[INFO] Including com.esotericsoftware:kryo:jar:5.0.0-RC4 in the shaded jar.
[INFO] Including com.esotericsoftware:reflectasm:jar:1.11.8 in the shaded jar.
[INFO] Including com.esotericsoftware:minlog:jar:1.3.1 in the shaded jar.
[INFO] Including org.rocksdb:rocksdbjni:jar:6.2.4 in the shaded jar.
[INFO] Including org.jooq:jooq:jar:3.12.3 in the shaded jar.
[INFO] Including org.reactivestreams:reactive-streams:jar:1.0.2 in the shaded jar.
[INFO] Including javax.xml.bind:jaxb-api:jar:2.3.0 in the shaded jar.
[INFO] Including eu.fasten.analyzer:base-analyzer:jar:0.0.1-SNAPSHOT in the shaded jar.
[INFO] Including com.ibm.wala:com.ibm.wala.core:jar:1.5.5 in the shaded jar.
[INFO] Including com.ibm.wala:com.ibm.wala.util:jar:1.5.5 in the shaded jar.
[INFO] Including com.ibm.wala:com.ibm.wala.shrike:jar:1.5.5 in the shaded jar.
[INFO] Including info.picocli:picocli:jar:4.0.4 in the shaded jar.
[INFO] Including org.gradle:gradle-tooling-api:jar:4.3 in the shaded jar.
[INFO] Including org.slf4j:slf4j-simple:jar:1.7.30 in the shaded jar.
[INFO] Including org.slf4j:slf4j-api:jar:1.7.30 in the shaded jar.
[INFO] Including org.objenesis:objenesis:jar:2.6 in the shaded jar.
[WARNING] Discovered module-info.class. Shading will break its strong encapsulation.
[WARNING] picocli-4.0.4.jar, base-analyzer-0.0.1-SNAPSHOT.jar define 1 overlapping classes: 
[WARNING]   - META-INF.versions.9.module-info
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/
[INFO] Attaching shaded artifact.
[INFO] 
[INFO] --- maven-install-plugin:2.4:install (default-install) @ javacg-wala ---
[INFO] Installing /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/target/javacg-wala-0.0.1-SNAPSHOT.jar to /home/tmortagne/.m2/repository/eu/fasten/analyzer/javacg-wala/0.0.1-SNAPSHOT/javacg-wala-0.0.1-SNAPSHOT.jar
[INFO] Installing /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/pom.xml to /home/tmortagne/.m2/repository/eu/fasten/analyzer/javacg-wala/0.0.1-SNAPSHOT/javacg-wala-0.0.1-SNAPSHOT.pom
[INFO] Installing /media/DATA/xwiki/xwikisas/research/FASTEN/github/fasten-fork/analyzer/javacg-wala/docker/plugins/javacg-wala-0.0.1-SNAPSHOT-with-dependencies.jar to /home/tmortagne/.m2/repository/eu/fasten/analyzer/javacg-wala/0.0.1-SNAPSHOT/javacg-wala-0.0.1-SNAPSHOT-with-dependencies.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  9.340 s
[INFO] Finished at: 2020-06-08T15:44:56+02:00
[INFO] ------------------------------------------------------------------------

Built with Maven 3.6.3.

KnowledgeBase/REST API integration tests

PostgreSQL integration tests (#92)
~~RESTeasy~~ Tomcat integration tests (changed in #123)

Performance measure for plugins

Is your feature request related to a problem? Please describe.

There is no simple way to measure performance of plugins as the only timestamp available in plugin output messages corresponds to the time the record was produced, but not when the input record was consumed.

Describe the solution you'd like

A simple solution to the problem would be adding a field to the output and error messages that contains a timestamp of the moment when the input record was consumed by the plugin.

Additional context

The only place where something similar is available is the log of the FASTEN server, where processing time is displayed. Adding consumed_at field to the output message would allow to measure performance of plugins without the need of aggregating logs across multiple docker containers and would also allow greater flexibility as both consumed_at and created_at fields will be available, compared to only record processing time in the server logs.

Switch to Spring for the REST API

As far as the REST API is concerned, there are still a few issues:

A couple of timeout #86
CDI not working (needed for unit testing) #87
Unsupported concurrent requests #93

All (or part of) these problems could be solved together by switching from RESTeasy to Spring (especially for Spring DI).

This change would also affect integration tests #89 and Dockerization #101.

Improve error handling in OPAL plugin

Is your feature request related to a problem? Please describe.

Currently it's impossible to differentiate between cases when an artifact couldn't be found in the Maven repository and when an artefact couldn't be written/read from the filesystem or any other similar type of error

Describe the solution you'd like

Make OPAL throw FileNotFoundException when an artifact couldn't be found on Maven and a RuntimeException in all other cases.

Additional context

It's merely impossible to tell those errors apart except for going through error logs manually coordinate by coordinate and checking if the file is present on Maven. Error caused by inability to write/read or any other similar errors skew the statistics of FileNotFoundException.

GraphMavenResolver returns incorrect dependencies

In the output of GraphMavenResolver.resolveFullDependencySet() for it.unimi.dsi / dsiutils / 2.2.2 (compile scope) we find hundreds of dependencies (see below). The actual dependencies (at the current timestamp) are

checker-qual-3.5.0.jar
commons-collections-20040616.jar
commons-configuration-1.10.jar
commons-io-2.8.0.jar
commons-lang-2.6.jar
commons-logging-1.1.1.jar
commons-math3-3.6.1.jar
error_prone_annotations-2.3.4.jar
failureaccess-1.0.1.jar
fastutil-8.4.3.jar
guava-30.0-jre.jar
j2objc-annotations-1.3.jar
jsap-2.1.jar
jsr305-3.0.2.jar
listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
slf4j-api-1.7.30.jar

that is, about a dozen.

cos:cos:05Nov2002
org.jruby:jruby:1.0.1
javax.activation:activation:1.1
jline:jline:1.0
poi:poi:2.5.1
javax.xml:jsr173:1.0
commons-net:commons-net:2.2
jetty:org.mortbay.jetty:5.1.4
org.apache.tiles:tiles-jsp:2.0.6
javax.ejb:ejb:3.0
org.codehaus.cargo:cargo-core-container-jo:0.8
cglib:cglib:2.1_3
dom4j:dom4j:1.6.1
backport-util-concurrent:backport-util-concurrent:3.0
hibernate:hibernate:2.1.8
org.aspectj:aspectjweaver:1.6.1
org.apache.xmlgraphics:batik-dom:1.7
axion:axion:1.0-M3-dev
msv:relaxngDatatype:20030807
mondrian:mondrian:2.3.2.8944
commons-lang:commons-lang:2.6
javax.servlet:jstl:1.1.0
bsh:bsh:1.2b3
org.apache.lucene:lucene-core:2.0.0
commons-io:commons-io:2.4
com.ibm.icu:icu4j:2.6.1
activation:activation:1.0.2
org.codehaus.cargo:cargo-core-container-jetty:0.8
jfree:jfreechart:[1.0.0,)
org.apache.tiles:tiles-api:2.0.6
bsf:bsf:2.4.0
velocity-tools:velocity-tools-view:1.4
werken-xpath:werken-xpath:0.9.4
avalon-framework:avalon-framework:4.1.3
javax.jts:jts:1.0
org.slf4j:slf4j-jdk14:1.7.6
org.apache.xmlgraphics:batik-svggen:1.7
com.thoughtworks.xstream:xstream:1.4.2
ch.qos.logback:logback-classic:1.1.2
qdox:qdox:1.5
org.jboss.logging:jboss-logging-spi:2.1.2.GA
org.jboss.netty:netty:3.2.7.Final
org.springframework:spring-web:2.0.2
dbunit:dbunit:2.1
checkstyle:checkstyle-optional:4.3
com.ibatis:ibatis2:2.3.0.677
org.apache.xmlgraphics:batik-gvt:1.7
${pom.groupId}:javax.servlet:1.0.0
emma:emma_ant:2.1.5320
commons-collections:commons-collections-testframework:3.2.1
xdoclet:xjavadoc:1.1
junit-addons:junit-addons:1.4
ehcache:ehcache:1.2
org.codehaus.jettison:jettison:1.2
xmlpull:xmlpull:1.1.3.1
howl:howl-logger:0.1.11
msv:xsdlib:20030807
oro:oro:2.0.8
org.springframework:spring-webmvc:2.0.2
castor:castor:0.9.9.0-pre
commons-cli:commons-cli:1.0
postgresql:postgresql:8.4-701.jdbc4
asm:asm-attrs:2.2
org.apache.xmlgraphics:batik-script:1.7
com.caucho:hessian:3.1.3
radeox:radeox:0.9
org.slf4j:slf4j-simple:1.3.1
xerces:xercesImpl:2.4.0
org.jdom:jaxen-core:1.0-FCS
org.springframework:spring:2.5.6
stax:stax-api:1.0
org.aspectj:aspectjrt:1.6.1
org.jdom:saxpath:1.0-FCS
cglib:cglib-full:2.0.2
org.apache.geronimo.specs:geronimo-j2ee-connector_1.5_spec:1.0
aopalliance:aopalliance:1.0
jmock:jmock-cglib:1.2.0
org.apache.xmlgraphics:batik-awt-util:1.7
org.apache.mina:mina-filter-ssl:1.1.7
javax.mail:mail:1.4
net.sf.kxml:kxml2-min:2.3.0
xerces:xmlParserAPIs:2.6.2
org.objectweb.carol:carol:2.0.5
commons-beanutils:commons-beanutils-core:1.7.0
jboss:jboss-minimal:4.0.2
jstl:jstl:1.0.6
javax.transaction:jta:1.1
org.mockito:mockito-all:1.8.2
javax.xml:jaxrpc-api:1.1
jexcelapi:jxl:2.6.6
mx4j:mx4j:3.0.2
${pom.groupId}:org.osgi.core:1.4.0
taglibs:standard:1.1.2
poi:poi-2.0-final:20040126
org.slf4j:nlog4j:1.2.24
org.springframework:spring-aspects:2.0.2
commons-pool:commons-pool:1.4
regexp:regexp:1.3
org.codehaus.cargo:cargo-core-container-orion:0.8
org.apache.felix:org.osgi.core:1.4.0
org.apache.ant:ant-testutil:1.7.0
jboss:jboss-system:4.0.2
com.tonicsystems:jarjar:0.6
org.apache.geronimo.specs:geronimo-j2ee-deployment_1.1_spec:1.0
org.hibernate:hibernate-entitymanager:3.3.2.GA
org.springframework:spring-core:2.0.2
commons-beanutils:commons-beanutils:1.8.3
slide:webdavlib:2.0
javacc:javacc:3.2
org.apache.poi:poi:3.0.1-FINAL
org.fusesource.jansi:jansi:1.6
com.sun.jmx:jmxri:1.2.1
jdbm:jdbm:1.0
org.apache.felix:org.apache.felix.main:2.0.2
javax.resource:connector:1.0
cglib:cglib-nodep:2.2
jboss:jboss-j2se:200504122039
emma:emma:2.1.5320
org.testng:testng:6.5.2
org.bouncycastle:bcprov-jdk14:1.45
org.codehaus.janino:commons-compiler:2.6.1
org.apache.jackrabbit:jackrabbit-jcr-commons:1.5.2
javax.sql:jdbc-stdext:2.0
org.codehaus.cargo:cargo-core-api-util:0.8
commons-attributes:commons-attributes-api:2.2
opensymphony:oscache:2.1
org.apache.ant:ant-nodeps:1.7.1
org.apache.commons:commons-compress:1.1
javax.security:jacc:1.0
antlr:antlr:2.7.2
rhino:js:1.6R2
commons-jxpath:commons-jxpath:1.3
${pom.groupId}:org.apache.felix.shell:1.4.1
hibernate:antlr:2.7.5H3
org.springframework:spring-dao:2.0.2
aspectj:aspectjweaver:1.5.3
htmlunit:htmlunit:1.8
com.martiansoftware:jsap:2.1
com.icegreen:greenmail:1.3
tomcat:naming-java:5.0.28
httpunit:httpunit:1.6.1
org.codehaus.jcsp:jcsp:1.1-rc5
org.springframework:spring-aop:2.0.2
javax.xml.soap:saaj-api:1.3
bouncycastle:bcmail-jdk14:138
xom:xom:1.0b3
org.apache.felix:org.osgi.compendium:1.4.0
servletapi:servletapi:2.3
org.freemarker:freemarker:2.3.14
org.apache.xmlgraphics:batik-anim:1.7
groovy:groovy:1.0
org.codehaus.groovy:groovy:1.5.6
proxool:proxool:0.8.3
commons-primitives:commons-primitives:1.0
jboss:javassist:3.3.ga
com.caucho:burlap:2.1.12
ant:ant:1.5.1
org.codehaus.cargo:cargo-core-api-module:0.8
org.subethamail:subethasmtp:2.1.0
xpp3:xpp3:1.1.3.3
radeox:radeox-oro:0.9
opensymphony:quartz-all:1.6.0
org.apache.commons:commons-vfs2:2.0
commons-validator:commons-validator:1.3.1
org.springframework:spring-beans:2.0.2
org.codehaus.cargo:cargo-core-container-geronimo:0.8
jaxen:jaxen:1.1-beta-6
org.apache.bsf:bsf-api:3.1
org.codehaus.jsr166-mirror:jsr166y:1.7.0
commons-collections:commons-collections:20040616
commons-jelly:commons-jelly-tags-xml:1.0
commons-jelly:commons-jelly-tags-log:1.0
com.google.inject:guice:2.0
net.sourceforge.jexcelapi:jxl:2.6
org.slf4j:integration:1.7.6
org.codehaus.cargo:cargo-ant:0.8
com.mockrunner:mockrunner-jdk1.3-j2ee1.3:0.4
javax.resource:connector-api:1.5
xalan:xalan:2.5.1
org.codehaus.gpars:gpars:1.0.0
${pom.groupId}:org.apache.felix.shell.tui:1.4.1
org.codehaus.cargo:cargo-core-uberjar:0.8
jtidy:jtidy:4aug2000r7-dev
velocity:velocity-dep:1.4
org.jdom:jaxen-jdom:1.0-FCS
org.apache.mina:mina-integration-jmx:1.1.7
com.servlets:cos:05Nov2002
commons-configuration:commons-configuration:1.8
commons-modeler:commons-modeler:2.0
org.codehaus.cargo:cargo-core-api-generic:0.8
com.google.guava:guava:18.0
log4j:log4j:1.2.17
junitperf:junitperf:1.8
xerces:xerces-impl:2.6.2
org.apache.shale:shale-test:1.0.4
org.springframework:spring-jpa:2.0.2
org.apache.xmlgraphics:batik-bridge:1.7
geronimo-spec:geronimo-spec-jta:1.0.1B-rc2
com.jcraft:jsch:0.1.42
jasperreports:jasperreports:2.0.5
org.codehaus.cargo:cargo-core-container-tomcat:0.8
${pom.groupId}:org.apache.felix.bundlerepository:1.4.2
ch.qos.logback:logback-core:1.1.2
logkit:logkit:1.0.1
commons-jelly:commons-jelly-tags-junit:1.0
com.oracle:oc4j:1.0
stax:stax-ri:1.0
jotm:jotm_iiop_stubs:2.0.10
xmlunit:xmlunit:1.1
net.sf.kxml:kxml2:2.3.0
org.slf4j:jul-to-slf4j:1.7.6
org.apache.ibatis:ibatis-sqlmap:2.3.4.726
org.apache.geronimo.specs:geronimo-ejb_2.1_spec:1.0
org.hamcrest:hamcrest-core:1.3
org.apache.ant:ant-antlr:1.8.4
com.cenqua.clover:clover:1.3.13
org.apache.jackrabbit:jackrabbit-webdav:1.5.2
commons-attributes:commons-attributes-compiler:2.2
org.springframework:spring-remoting:1.2.8
javax.jms:jms:1.1
com.sun.jdmk:jmxtools:1.2.1
jgroups:jgroups-all:2.4.1
org.apache.xmlgraphics:batik-css:1.7
asm:asm-commons:2.2.3
com.bea.xml:jsr173-ri:1.0
org.apache.felix:org.osgi.foundation:1.2.0
c3p0:c3p0:0.9.1.2
jboss:jboss-archive-browsing:5.0.0alpha-200607201-119
woodstox:wstx-asl:3.2.2
org.apache.directory.server:apacheds-core:1.0-RC3
com.h2database:h2:1.2.132
jdom:jdom:1.0
com.lowagie:itext:2.0.7
tomcat:catalina:5.5.23
jmock:jmock:1.0.0
com.bea.wlplatform:commonj-twm:1.1
org.acegisecurity:acegi-security:1.0.3
stax:stax:1.2.0
org.apache.xmlgraphics:batik-xml:1.7
org.codehaus.cargo:cargo-core-container-jboss:0.8
com.google.code.findbugs:jsr305:1.3.9
groovy:groovy-all-minimal:1.0
org.apache.geronimo.specs:geronimo-servlet_2.4_spec:1.0
cactus:cactus:12-1.4.1
org.beanshell:bsh:2.0b4
org.codehaus.cargo:cargo-core-container-weblogic:0.8
joda-time:joda-time:1.6
org.hibernate:ejb3-persistence:1.0.1.GA
org.springframework:spring-context:2.0.2
commons-javaflow:commons-javaflow:20060411
org.codehaus.cargo:cargo-core-container-resin:0.8
struts:struts:1.2.9
org.slf4j:slf4j-ext:1.7.6
mockobjects:mockobjects-core:0.09
aspectj:aspectjrt:1.5.3
junit:junit:4.12-beta-1
xpp3:xpp3_min:1.1.4c
nekohtml:nekohtml:0.9.5
pull-parser:pull-parser:2
org.slf4j:slf4j-log4j12:1.7.6
com.thoughtworks.qdox:qdox:1.12
com.google.protobuf:protobuf-java:2.4.1
org.springframework:spring-support:1.2.8
org.mockito:mockito-core:1.9.0
org.apache.commons:commons-math3:3.3
com.oracle:toplink-essentials:2.41
org.codehaus.groovy:groovy-all:2.0.7
org.easymock:easymock:3.1
org.objenesis:objenesis:1.0
quartz:quartz:1.6.0
org.codehaus.cargo:cargo-core-api-container:0.8
xml-apis:xml-apis-ext:1.3.04
org.springframework:spring-jdbc:2.0.2
net.sf.hibernate:hibernate:2.1.8
saxpath:saxpath:1.0-FCS
javax.servlet:servlet-api:2.3
xml-resolver:xml-resolver:1.2
org.springframework:spring-agent:2.0.2
javax.jcr:jcr:1.0
commons-codec:commons-codec:1.5
org.codehaus.woodstox:wstx-asl:3.2.7
mockobjects:mockobjects-jdk1.4-j2ee1.3:0.09
org.apache.xmlgraphics:batik-util:1.7
javassist:javassist:3.4.GA
com.experlog:xapool:1.5.0
org.easymock:easymockclassextension:3.1
idb:idb:3.26
org.springframework:spring-mock:2.0.2
forehead:forehead:1.0-beta-5
commons-digester:commons-digester:1.8.1
xerces:xerces:2.4.0
com.oracle.toplink:toplink:10.1.3
org.apache.ant:ant-junit:1.8.4
javax.faces:jsf-api:1.1
org.jdom:jdom:1.1
jboss:jboss-jee:4.2.0.GA
xml-apis:xml-apis:1.0.b2
org.apache.geronimo.specs:geronimo-jms_1.1_spec:1.0
commons-dbcp:commons-dbcp:1.2.2
velocity:velocity:1.5
gsbase:gsbase:2.0.1
commons-jelly:commons-jelly:1.0
org.apache.commons:commons-jexl:2.1.1
openejb:openejb-loader:1.0
itext:itext:1.3
ch.qos.cal10n:cal10n-api:0.8.1
org.yaml:snakeyaml:1.6
asm:asm-tree:2.2.3
commons-logging:commons-logging:1.1.1
bouncycastle:bcprov-jdk14:138
org.apache.mina:mina-core:
jruby:jruby:0.9.2
commons-fileupload:commons-fileupload:1.2
jotm:jotm_jrmp_stubs:2.0.10
org.apache.xmlgraphics:batik-parser:1.7
org.apache.velocity:velocity:1.6.2
org.slf4j:log4j-over-slf4j:1.7.6
org.apache.openejb:javaee-api:5.0-2
org.ccil.cowan.tagsoup:tagsoup:0.9.7
swarmcache:swarmcache:1.0RC2
jaxme:jaxme-api:0.3
com.jamonapi:jamon:2.4
org.hibernate:hibernate-annotations:3.3.1.GA
${pom.groupId}:org.apache.felix.framework:2.0.2
jboss:jboss-cache:1.2.2
org.hibernate:hibernate:3.2.6.ga
org.json:json:20080701
org.apache.ant:ant-launcher:1.7.1
xstream:xstream:1.2
ant:ant-junit:1.6.5
org.bouncycastle:bcpg-jdk14:1.45
javax.portlet:portlet-api:1.0
it.unimi.dsi:fastutil:6.5.15
eclipse:jdtcore:[3.1.0,)
jfree:jcommon:[1.0.0,)
ch.qos.logback:logback-classic.jar:2.2.2
org.apache.geronimo.specs:geronimo-jta_1.0.1B_spec:1.0
ant:ant-nodeps:1.6.2
portlet-api:portlet-api:1.0
groovy:groovy-all:1.0-beta-10
openejb:openejb-core:1.0
org.mockejb:mockejb:0.6-beta2
commons-vfs:commons-vfs:1.0
mysql:mysql-connector-java:5.1.9
net.sf.ehcache:ehcache:1.5.0
javax.servlet:jsp-api:2.0
asm:asm-analysis:2.2
ant:ant-trax:1.6.2
com.keypoint:png-encoder:1.5
checkstyle:checkstyle:4.3
org.apache.ant:ant:1.7.1
javax.persistence:persistence-api:1.0
org.codehaus.janino:janino:2.6.1
commons-httpclient:commons-httpclient:2.0.2
mx4j:mx4j-jmx:2.1.1
javax.jdo:jdo2-api:2.0
com.beust:jcommander:1.12
commons-logging:commons-logging-api:1.1
org.apache.ivy:ivy:2.2.0
net.sf.jsr107cache:jsr107cache:1.0
commons-beanutils:commons-beanutils-bean-collections:1.7.0
org.slf4j:slf4j-api:1.7.7
commons-discovery:commons-discovery:20030211.213356
asm:asm-util:2.2.3
ant:ant-launcher:1.6.2
jotm:jotm:2.0.10
freemarker:freemarker:2.3.8
tomcat:naming-common:5.0.28
com.megginson.sax:xml-writer:0.2
org.hibernate:hibernate-commons-annotations:3.0.0.ga
org.multiverse:multiverse-beta:0.7-RC-1
com.jcraft:jzlib:1.0.7
org.apache.xmlgraphics:batik-ext:1.7
org.apache.directory.server:apacheds-core-shared:1.0-RC3
org.apache.tiles:tiles-core:2.0.6
concurrent:concurrent:1.3.4
asm:asm:2.2.3
org.apache.axis:axis:1.4
cglib-nodep:cglib-nodep:2.1_3
javax.j2ee:j2ee:1.4
org.apache.xmlgraphics:batik-svg-dom:1.7
hessian:hessian:3.0.20
jboss:jboss-common:4.0.2
velocity-tools:velocity-tools-generic:1.4
cactus:cactus-ant:1.4.1
org.apache.xmlgraphics:batik-js:1.7
hsqldb:hsqldb:1.8.0.7
commons-jexl:commons-jexl:1.0
cas:casclient:2.0.11

Switching to fastutils in the metadata plugin

Is your feature request related to a problem? Please describe.

Generated call graphs are quite big and inserting them into the database takes quite some time.

Describe the solution you'd like

Using data structures with fast access (e.g. fastutil) might help to improve the performance.

Additional context

No additional context

Files without documentation

This is a list of files without documentation, which helps us find the files and improve their docs.
I can also help with the documentation.

fasten/core/src/main/java/eu/fasten/core/query/Query.java

Line 7 in b0421f2
fasten/core/src/main/java/eu/fasten/core/index/FenwickTree.java

Line 24 in b0421f2
fasten/pipeliner/src/main/java/eu/fasten/pipeliner/Pipeliner.java

Line 36 in b0421f2
fasten/server/src/main/java/eu/fasten/server/FastenServer.java

Line 34 in b0421f2
fasten/core/src/main/java/eu/fasten/core/index/KBStats.java

Line 38 in b0421f2
fasten/core/src/main/java/eu/fasten/core/index/BVGraphSerializer.java

Line 29 in 85a013c

Merge Indexer and Metadata Plugin

Unite Indexer and Metadata Plugin into Database Plugin which consumes ERCG from Kafka, then puts it into metadata database (PostgreSQL) and using obtained indexes puts the graph into graph database (RocksDB).

Refactor MetadataPlugin to have a KnowledgeBase instance and all required indexes from PostgreSQL
Refactor KnowledgeBase to receive all indexes and use them instead of generating own

Wiki uses mvn-crawler Docker image

The Development Environment wiki page makes use of the mvn-crawler Docker image in its step-by-step guide.
Yet, its Dockerfile has been removed in fasten-project/mvn-crawler@0ea00f8#diff-1eef4e1e3eb8b97aec14b6bbd5332799.

javacg-opal: mvn install has test failures on Windows 10

JAVA_HOME is set c:/Program Files/OpenJDK/openjdk-11.0.7_10

Please see trace attached.
trace.txt

REST API pagination

This is how I see it:

Method	Pros	Cons
Limit-offset	Simple Universal	Result inconsistency Inefficacy for high offset values
Cursors	Universal Result consistency	Not scalable Memory Locks Stateful
Keyset pagination	Result consistency Offset efficiency	Requires indexed columns ✔️ No random access (no offsets)

If we don't mind about inconsistency and performance for high offset values, I'd keep it simple and I'd go for the classic limit & offset approach. @gousiosg any take on this?

Implement support for connecting multiple postgres databases to the FASTEN Server

Currently, FASTEN supports connecting to only one database at a time. On the other hand, Quality Analyzer requires to use multiple databases.

Implement SWH compatible SHA-1s

Is your feature request related to a problem? Please describe.

We currently compute by SHA1s by calling the SHA1 function on the file contents. To link with SWH's software archive, we need to compute git compatible SHA1s

Describe the solution you'd like

Add a column to the files table (e.g. swh_sha1) and fill it in with git salted SHA1s. As per @zvr:

Git computes the SHA of a file object (blob) by "salting" (prepending) to its contents the word "blob", a space, the size of the file in decimal and a NUL byte.

$ echo 'Hello, World!' | git hash-object -w --stdin
8ab686eafeb1f44702738c8b0f24f2567c36da6d
$ echo -e 'blob 14\0Hello, World!' | shasum
8ab686eafeb1f44702738c8b0f24f2567c36da6d

Additional context

Information about SWH persistent ids

3 tests for fasten-core fail on Windows 10

$ mvn install

"%HOME%" in the following should be read as user's homedir.

Tests in error:
`
testMediumIndex(eu.fasten.core.index.IndexerTest): Unable to delete file: %HOME%\AppData\Local\Temp\Indexer8430612523326898716\LOCK

testSmallIndex(eu.fasten.core.index.IndexerTest): Unable to delete file: %HOME%\AppData\Local\Temp\Indexer10475225208700705875\LOCK

testLargeIndex(eu.fasten.core.index.IndexerTest): Unable to delete file: %HOME%\AppData\Local\Temp\Indexer7022387531492936798\LOCK`

Stacktrace on 1, others are very similar:

testMediumIndex(eu.fasten.core.index.IndexerTest) Time elapsed: 1.338 sec <<< ERROR! java.io.IOException: Unable to delete file: %HOME%\AppData\Local\Temp\Indexer8430612523326898716\LOCK at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2400) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1721) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1617) at eu.fasten.core.index.IndexerTest.testKnowledgeBase(IndexerTest.java:210) at eu.fasten.core.index.IndexerTest.testMediumIndex(IndexerTest.java:226) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Allow passing a custom maven repository to javacg analyzers

javacg-wala and javacg-opal used a hardcoded https://repo.maven.apache.org/maven2/ for the repository when passing coordinates.

Would be very useful to allow passing an list of alternative Maven repositories to look at to find the artifact and its dependencies.

GraphMavenResolver returns version ranges instead of versions

In the output of GraphMavenResolver.resolveFullDependencySet() for it.unimi.dsi / dsiutils / 2.2.2 (compile scope) we find dependencies such as

jfree:jfreechart:[1.0.0,)

These are not resolved dependencies—resolved dependencies are artifacts. In particular, these outputs cannot be used with DatabaseMerger.

The entire returned set, displayed using toMavenCoordinate(), is below:

cos:cos:05Nov2002
org.jruby:jruby:1.0.1
javax.activation:activation:1.1
jline:jline:1.0
poi:poi:2.5.1
javax.xml:jsr173:1.0
commons-net:commons-net:2.2
jetty:org.mortbay.jetty:5.1.4
org.apache.tiles:tiles-jsp:2.0.6
javax.ejb:ejb:3.0
org.codehaus.cargo:cargo-core-container-jo:0.8
cglib:cglib:2.1_3
dom4j:dom4j:1.6.1
backport-util-concurrent:backport-util-concurrent:3.0
hibernate:hibernate:2.1.8
org.aspectj:aspectjweaver:1.6.1
org.apache.xmlgraphics:batik-dom:1.7
axion:axion:1.0-M3-dev
msv:relaxngDatatype:20030807
mondrian:mondrian:2.3.2.8944
commons-lang:commons-lang:2.6
javax.servlet:jstl:1.1.0
bsh:bsh:1.2b3
org.apache.lucene:lucene-core:2.0.0
commons-io:commons-io:2.4
com.ibm.icu:icu4j:2.6.1
activation:activation:1.0.2
org.codehaus.cargo:cargo-core-container-jetty:0.8
jfree:jfreechart:[1.0.0,)
org.apache.tiles:tiles-api:2.0.6
bsf:bsf:2.4.0
velocity-tools:velocity-tools-view:1.4
werken-xpath:werken-xpath:0.9.4
avalon-framework:avalon-framework:4.1.3
javax.jts:jts:1.0
org.slf4j:slf4j-jdk14:1.7.6
org.apache.xmlgraphics:batik-svggen:1.7
com.thoughtworks.xstream:xstream:1.4.2
ch.qos.logback:logback-classic:1.1.2
qdox:qdox:1.5
org.jboss.logging:jboss-logging-spi:2.1.2.GA
org.jboss.netty:netty:3.2.7.Final
org.springframework:spring-web:2.0.2
dbunit:dbunit:2.1
checkstyle:checkstyle-optional:4.3
com.ibatis:ibatis2:2.3.0.677
org.apache.xmlgraphics:batik-gvt:1.7
${pom.groupId}:javax.servlet:1.0.0
emma:emma_ant:2.1.5320
commons-collections:commons-collections-testframework:3.2.1
xdoclet:xjavadoc:1.1
junit-addons:junit-addons:1.4
ehcache:ehcache:1.2
org.codehaus.jettison:jettison:1.2
xmlpull:xmlpull:1.1.3.1
howl:howl-logger:0.1.11
msv:xsdlib:20030807
oro:oro:2.0.8
org.springframework:spring-webmvc:2.0.2
castor:castor:0.9.9.0-pre
commons-cli:commons-cli:1.0
postgresql:postgresql:8.4-701.jdbc4
asm:asm-attrs:2.2
org.apache.xmlgraphics:batik-script:1.7
com.caucho:hessian:3.1.3
radeox:radeox:0.9
org.slf4j:slf4j-simple:1.3.1
xerces:xercesImpl:2.4.0
org.jdom:jaxen-core:1.0-FCS
org.springframework:spring:2.5.6
stax:stax-api:1.0
org.aspectj:aspectjrt:1.6.1
org.jdom:saxpath:1.0-FCS
cglib:cglib-full:2.0.2
org.apache.geronimo.specs:geronimo-j2ee-connector_1.5_spec:1.0
aopalliance:aopalliance:1.0
jmock:jmock-cglib:1.2.0
org.apache.xmlgraphics:batik-awt-util:1.7
org.apache.mina:mina-filter-ssl:1.1.7
javax.mail:mail:1.4
net.sf.kxml:kxml2-min:2.3.0
xerces:xmlParserAPIs:2.6.2
org.objectweb.carol:carol:2.0.5
commons-beanutils:commons-beanutils-core:1.7.0
jboss:jboss-minimal:4.0.2
jstl:jstl:1.0.6
javax.transaction:jta:1.1
org.mockito:mockito-all:1.8.2
javax.xml:jaxrpc-api:1.1
jexcelapi:jxl:2.6.6
mx4j:mx4j:3.0.2
${pom.groupId}:org.osgi.core:1.4.0
taglibs:standard:1.1.2
poi:poi-2.0-final:20040126
org.slf4j:nlog4j:1.2.24
org.springframework:spring-aspects:2.0.2
commons-pool:commons-pool:1.4
regexp:regexp:1.3
org.codehaus.cargo:cargo-core-container-orion:0.8
org.apache.felix:org.osgi.core:1.4.0
org.apache.ant:ant-testutil:1.7.0
jboss:jboss-system:4.0.2
com.tonicsystems:jarjar:0.6
org.apache.geronimo.specs:geronimo-j2ee-deployment_1.1_spec:1.0
org.hibernate:hibernate-entitymanager:3.3.2.GA
org.springframework:spring-core:2.0.2
commons-beanutils:commons-beanutils:1.8.3
slide:webdavlib:2.0
javacc:javacc:3.2
org.apache.poi:poi:3.0.1-FINAL
org.fusesource.jansi:jansi:1.6
com.sun.jmx:jmxri:1.2.1
jdbm:jdbm:1.0
org.apache.felix:org.apache.felix.main:2.0.2
javax.resource:connector:1.0
cglib:cglib-nodep:2.2
jboss:jboss-j2se:200504122039
emma:emma:2.1.5320
org.testng:testng:6.5.2
org.bouncycastle:bcprov-jdk14:1.45
org.codehaus.janino:commons-compiler:2.6.1
org.apache.jackrabbit:jackrabbit-jcr-commons:1.5.2
javax.sql:jdbc-stdext:2.0
org.codehaus.cargo:cargo-core-api-util:0.8
commons-attributes:commons-attributes-api:2.2
opensymphony:oscache:2.1
org.apache.ant:ant-nodeps:1.7.1
org.apache.commons:commons-compress:1.1
javax.security:jacc:1.0
antlr:antlr:2.7.2
rhino:js:1.6R2
commons-jxpath:commons-jxpath:1.3
${pom.groupId}:org.apache.felix.shell:1.4.1
hibernate:antlr:2.7.5H3
org.springframework:spring-dao:2.0.2
aspectj:aspectjweaver:1.5.3
htmlunit:htmlunit:1.8
com.martiansoftware:jsap:2.1
com.icegreen:greenmail:1.3
tomcat:naming-java:5.0.28
httpunit:httpunit:1.6.1
org.codehaus.jcsp:jcsp:1.1-rc5
org.springframework:spring-aop:2.0.2
javax.xml.soap:saaj-api:1.3
bouncycastle:bcmail-jdk14:138
xom:xom:1.0b3
org.apache.felix:org.osgi.compendium:1.4.0
servletapi:servletapi:2.3
org.freemarker:freemarker:2.3.14
org.apache.xmlgraphics:batik-anim:1.7
groovy:groovy:1.0
org.codehaus.groovy:groovy:1.5.6
proxool:proxool:0.8.3
commons-primitives:commons-primitives:1.0
jboss:javassist:3.3.ga
com.caucho:burlap:2.1.12
ant:ant:1.5.1
org.codehaus.cargo:cargo-core-api-module:0.8
org.subethamail:subethasmtp:2.1.0
xpp3:xpp3:1.1.3.3
radeox:radeox-oro:0.9
opensymphony:quartz-all:1.6.0
org.apache.commons:commons-vfs2:2.0
commons-validator:commons-validator:1.3.1
org.springframework:spring-beans:2.0.2
org.codehaus.cargo:cargo-core-container-geronimo:0.8
jaxen:jaxen:1.1-beta-6
org.apache.bsf:bsf-api:3.1
org.codehaus.jsr166-mirror:jsr166y:1.7.0
commons-collections:commons-collections:20040616
commons-jelly:commons-jelly-tags-xml:1.0
commons-jelly:commons-jelly-tags-log:1.0
com.google.inject:guice:2.0
net.sourceforge.jexcelapi:jxl:2.6
org.slf4j:integration:1.7.6
org.codehaus.cargo:cargo-ant:0.8
com.mockrunner:mockrunner-jdk1.3-j2ee1.3:0.4
javax.resource:connector-api:1.5
xalan:xalan:2.5.1
org.codehaus.gpars:gpars:1.0.0
${pom.groupId}:org.apache.felix.shell.tui:1.4.1
org.codehaus.cargo:cargo-core-uberjar:0.8
jtidy:jtidy:4aug2000r7-dev
velocity:velocity-dep:1.4
org.jdom:jaxen-jdom:1.0-FCS
org.apache.mina:mina-integration-jmx:1.1.7
com.servlets:cos:05Nov2002
commons-configuration:commons-configuration:1.8
commons-modeler:commons-modeler:2.0
org.codehaus.cargo:cargo-core-api-generic:0.8
com.google.guava:guava:18.0
log4j:log4j:1.2.17
junitperf:junitperf:1.8
xerces:xerces-impl:2.6.2
org.apache.shale:shale-test:1.0.4
org.springframework:spring-jpa:2.0.2
org.apache.xmlgraphics:batik-bridge:1.7
geronimo-spec:geronimo-spec-jta:1.0.1B-rc2
com.jcraft:jsch:0.1.42
jasperreports:jasperreports:2.0.5
org.codehaus.cargo:cargo-core-container-tomcat:0.8
${pom.groupId}:org.apache.felix.bundlerepository:1.4.2
ch.qos.logback:logback-core:1.1.2
logkit:logkit:1.0.1
commons-jelly:commons-jelly-tags-junit:1.0
com.oracle:oc4j:1.0
stax:stax-ri:1.0
jotm:jotm_iiop_stubs:2.0.10
xmlunit:xmlunit:1.1
net.sf.kxml:kxml2:2.3.0
org.slf4j:jul-to-slf4j:1.7.6
org.apache.ibatis:ibatis-sqlmap:2.3.4.726
org.apache.geronimo.specs:geronimo-ejb_2.1_spec:1.0
org.hamcrest:hamcrest-core:1.3
org.apache.ant:ant-antlr:1.8.4
com.cenqua.clover:clover:1.3.13
org.apache.jackrabbit:jackrabbit-webdav:1.5.2
commons-attributes:commons-attributes-compiler:2.2
org.springframework:spring-remoting:1.2.8
javax.jms:jms:1.1
com.sun.jdmk:jmxtools:1.2.1
jgroups:jgroups-all:2.4.1
org.apache.xmlgraphics:batik-css:1.7
asm:asm-commons:2.2.3
com.bea.xml:jsr173-ri:1.0
org.apache.felix:org.osgi.foundation:1.2.0
c3p0:c3p0:0.9.1.2
jboss:jboss-archive-browsing:5.0.0alpha-200607201-119
woodstox:wstx-asl:3.2.2
org.apache.directory.server:apacheds-core:1.0-RC3
com.h2database:h2:1.2.132
jdom:jdom:1.0
com.lowagie:itext:2.0.7
tomcat:catalina:5.5.23
jmock:jmock:1.0.0
com.bea.wlplatform:commonj-twm:1.1
org.acegisecurity:acegi-security:1.0.3
stax:stax:1.2.0
org.apache.xmlgraphics:batik-xml:1.7
org.codehaus.cargo:cargo-core-container-jboss:0.8
com.google.code.findbugs:jsr305:1.3.9
groovy:groovy-all-minimal:1.0
org.apache.geronimo.specs:geronimo-servlet_2.4_spec:1.0
cactus:cactus:12-1.4.1
org.beanshell:bsh:2.0b4
org.codehaus.cargo:cargo-core-container-weblogic:0.8
joda-time:joda-time:1.6
org.hibernate:ejb3-persistence:1.0.1.GA
org.springframework:spring-context:2.0.2
commons-javaflow:commons-javaflow:20060411
org.codehaus.cargo:cargo-core-container-resin:0.8
struts:struts:1.2.9
org.slf4j:slf4j-ext:1.7.6
mockobjects:mockobjects-core:0.09
aspectj:aspectjrt:1.5.3
junit:junit:4.12-beta-1
xpp3:xpp3_min:1.1.4c
nekohtml:nekohtml:0.9.5
pull-parser:pull-parser:2
org.slf4j:slf4j-log4j12:1.7.6
com.thoughtworks.qdox:qdox:1.12
com.google.protobuf:protobuf-java:2.4.1
org.springframework:spring-support:1.2.8
org.mockito:mockito-core:1.9.0
org.apache.commons:commons-math3:3.3
com.oracle:toplink-essentials:2.41
org.codehaus.groovy:groovy-all:2.0.7
org.easymock:easymock:3.1
org.objenesis:objenesis:1.0
quartz:quartz:1.6.0
org.codehaus.cargo:cargo-core-api-container:0.8
xml-apis:xml-apis-ext:1.3.04
org.springframework:spring-jdbc:2.0.2
net.sf.hibernate:hibernate:2.1.8
saxpath:saxpath:1.0-FCS
javax.servlet:servlet-api:2.3
xml-resolver:xml-resolver:1.2
org.springframework:spring-agent:2.0.2
javax.jcr:jcr:1.0
commons-codec:commons-codec:1.5
org.codehaus.woodstox:wstx-asl:3.2.7
mockobjects:mockobjects-jdk1.4-j2ee1.3:0.09
org.apache.xmlgraphics:batik-util:1.7
javassist:javassist:3.4.GA
com.experlog:xapool:1.5.0
org.easymock:easymockclassextension:3.1
idb:idb:3.26
org.springframework:spring-mock:2.0.2
forehead:forehead:1.0-beta-5
commons-digester:commons-digester:1.8.1
xerces:xerces:2.4.0
com.oracle.toplink:toplink:10.1.3
org.apache.ant:ant-junit:1.8.4
javax.faces:jsf-api:1.1
org.jdom:jdom:1.1
jboss:jboss-jee:4.2.0.GA
xml-apis:xml-apis:1.0.b2
org.apache.geronimo.specs:geronimo-jms_1.1_spec:1.0
commons-dbcp:commons-dbcp:1.2.2
velocity:velocity:1.5
gsbase:gsbase:2.0.1
commons-jelly:commons-jelly:1.0
org.apache.commons:commons-jexl:2.1.1
openejb:openejb-loader:1.0
itext:itext:1.3
ch.qos.cal10n:cal10n-api:0.8.1
org.yaml:snakeyaml:1.6
asm:asm-tree:2.2.3
commons-logging:commons-logging:1.1.1
bouncycastle:bcprov-jdk14:138
org.apache.mina:mina-core:
jruby:jruby:0.9.2
commons-fileupload:commons-fileupload:1.2
jotm:jotm_jrmp_stubs:2.0.10
org.apache.xmlgraphics:batik-parser:1.7
org.apache.velocity:velocity:1.6.2
org.slf4j:log4j-over-slf4j:1.7.6
org.apache.openejb:javaee-api:5.0-2
org.ccil.cowan.tagsoup:tagsoup:0.9.7
swarmcache:swarmcache:1.0RC2
jaxme:jaxme-api:0.3
com.jamonapi:jamon:2.4
org.hibernate:hibernate-annotations:3.3.1.GA
${pom.groupId}:org.apache.felix.framework:2.0.2
jboss:jboss-cache:1.2.2
org.hibernate:hibernate:3.2.6.ga
org.json:json:20080701
org.apache.ant:ant-launcher:1.7.1
xstream:xstream:1.2
ant:ant-junit:1.6.5
org.bouncycastle:bcpg-jdk14:1.45
javax.portlet:portlet-api:1.0
it.unimi.dsi:fastutil:6.5.15
eclipse:jdtcore:[3.1.0,)
jfree:jcommon:[1.0.0,)
ch.qos.logback:logback-classic.jar:2.2.2
org.apache.geronimo.specs:geronimo-jta_1.0.1B_spec:1.0
ant:ant-nodeps:1.6.2
portlet-api:portlet-api:1.0
groovy:groovy-all:1.0-beta-10
openejb:openejb-core:1.0
org.mockejb:mockejb:0.6-beta2
commons-vfs:commons-vfs:1.0
mysql:mysql-connector-java:5.1.9
net.sf.ehcache:ehcache:1.5.0
javax.servlet:jsp-api:2.0
asm:asm-analysis:2.2
ant:ant-trax:1.6.2
com.keypoint:png-encoder:1.5
checkstyle:checkstyle:4.3
org.apache.ant:ant:1.7.1
javax.persistence:persistence-api:1.0
org.codehaus.janino:janino:2.6.1
commons-httpclient:commons-httpclient:2.0.2
mx4j:mx4j-jmx:2.1.1
javax.jdo:jdo2-api:2.0
com.beust:jcommander:1.12
commons-logging:commons-logging-api:1.1
org.apache.ivy:ivy:2.2.0
net.sf.jsr107cache:jsr107cache:1.0
commons-beanutils:commons-beanutils-bean-collections:1.7.0
org.slf4j:slf4j-api:1.7.7
commons-discovery:commons-discovery:20030211.213356
asm:asm-util:2.2.3
ant:ant-launcher:1.6.2
jotm:jotm:2.0.10
freemarker:freemarker:2.3.8
tomcat:naming-common:5.0.28
com.megginson.sax:xml-writer:0.2
org.hibernate:hibernate-commons-annotations:3.0.0.ga
org.multiverse:multiverse-beta:0.7-RC-1
com.jcraft:jzlib:1.0.7
org.apache.xmlgraphics:batik-ext:1.7
org.apache.directory.server:apacheds-core-shared:1.0-RC3
org.apache.tiles:tiles-core:2.0.6
concurrent:concurrent:1.3.4
asm:asm:2.2.3
org.apache.axis:axis:1.4
cglib-nodep:cglib-nodep:2.1_3
javax.j2ee:j2ee:1.4
org.apache.xmlgraphics:batik-svg-dom:1.7
hessian:hessian:3.0.20
jboss:jboss-common:4.0.2
velocity-tools:velocity-tools-generic:1.4
cactus:cactus-ant:1.4.1
org.apache.xmlgraphics:batik-js:1.7
hsqldb:hsqldb:1.8.0.7
commons-jexl:commons-jexl:1.0
cas:casclient:2.0.11

Docker: NPE running docker.pkg.github.com/fasten-project/fasten/fasten.server:latest

Command: $ ~/repos/fasten $ docker run --network="host" --log-opt max-size=1g -d docker.pkg.github.com/fasten-project/fasten/fasten.server:latest -k localhost:30001

Log, NPE at line 255:
log.txt

GraphMavenResolver return empty version strings (which cause exceptions in DatabaseMerger)

In the output of GraphMavenResolver.resolveFullDependencySet() for it.unimi.dsi / dsiutils / 2.2.2 (compile scope) we find dependencies without versions, e.g., org.apache.mina:mina-core: (see below). These dependencies, when fed to DatabaseMerger, cause an ArrayIndexOutOfBoundException as the artifact name is parsed but the number of components is not checked.

cos:cos:05Nov2002
org.jruby:jruby:1.0.1
javax.activation:activation:1.1
jline:jline:1.0
poi:poi:2.5.1
javax.xml:jsr173:1.0
commons-net:commons-net:2.2
jetty:org.mortbay.jetty:5.1.4
org.apache.tiles:tiles-jsp:2.0.6
javax.ejb:ejb:3.0
org.codehaus.cargo:cargo-core-container-jo:0.8
cglib:cglib:2.1_3
dom4j:dom4j:1.6.1
backport-util-concurrent:backport-util-concurrent:3.0
hibernate:hibernate:2.1.8
org.aspectj:aspectjweaver:1.6.1
org.apache.xmlgraphics:batik-dom:1.7
axion:axion:1.0-M3-dev
msv:relaxngDatatype:20030807
mondrian:mondrian:2.3.2.8944
commons-lang:commons-lang:2.6
javax.servlet:jstl:1.1.0
bsh:bsh:1.2b3
org.apache.lucene:lucene-core:2.0.0
commons-io:commons-io:2.4
com.ibm.icu:icu4j:2.6.1
activation:activation:1.0.2
org.codehaus.cargo:cargo-core-container-jetty:0.8
jfree:jfreechart:[1.0.0,)
org.apache.tiles:tiles-api:2.0.6
bsf:bsf:2.4.0
velocity-tools:velocity-tools-view:1.4
werken-xpath:werken-xpath:0.9.4
avalon-framework:avalon-framework:4.1.3
javax.jts:jts:1.0
org.slf4j:slf4j-jdk14:1.7.6
org.apache.xmlgraphics:batik-svggen:1.7
com.thoughtworks.xstream:xstream:1.4.2
ch.qos.logback:logback-classic:1.1.2
qdox:qdox:1.5
org.jboss.logging:jboss-logging-spi:2.1.2.GA
org.jboss.netty:netty:3.2.7.Final
org.springframework:spring-web:2.0.2
dbunit:dbunit:2.1
checkstyle:checkstyle-optional:4.3
com.ibatis:ibatis2:2.3.0.677
org.apache.xmlgraphics:batik-gvt:1.7
${pom.groupId}:javax.servlet:1.0.0
emma:emma_ant:2.1.5320
commons-collections:commons-collections-testframework:3.2.1
xdoclet:xjavadoc:1.1
junit-addons:junit-addons:1.4
ehcache:ehcache:1.2
org.codehaus.jettison:jettison:1.2
xmlpull:xmlpull:1.1.3.1
howl:howl-logger:0.1.11
msv:xsdlib:20030807
oro:oro:2.0.8
org.springframework:spring-webmvc:2.0.2
castor:castor:0.9.9.0-pre
commons-cli:commons-cli:1.0
postgresql:postgresql:8.4-701.jdbc4
asm:asm-attrs:2.2
org.apache.xmlgraphics:batik-script:1.7
com.caucho:hessian:3.1.3
radeox:radeox:0.9
org.slf4j:slf4j-simple:1.3.1
xerces:xercesImpl:2.4.0
org.jdom:jaxen-core:1.0-FCS
org.springframework:spring:2.5.6
stax:stax-api:1.0
org.aspectj:aspectjrt:1.6.1
org.jdom:saxpath:1.0-FCS
cglib:cglib-full:2.0.2
org.apache.geronimo.specs:geronimo-j2ee-connector_1.5_spec:1.0
aopalliance:aopalliance:1.0
jmock:jmock-cglib:1.2.0
org.apache.xmlgraphics:batik-awt-util:1.7
org.apache.mina:mina-filter-ssl:1.1.7
javax.mail:mail:1.4
net.sf.kxml:kxml2-min:2.3.0
xerces:xmlParserAPIs:2.6.2
org.objectweb.carol:carol:2.0.5
commons-beanutils:commons-beanutils-core:1.7.0
jboss:jboss-minimal:4.0.2
jstl:jstl:1.0.6
javax.transaction:jta:1.1
org.mockito:mockito-all:1.8.2
javax.xml:jaxrpc-api:1.1
jexcelapi:jxl:2.6.6
mx4j:mx4j:3.0.2
${pom.groupId}:org.osgi.core:1.4.0
taglibs:standard:1.1.2
poi:poi-2.0-final:20040126
org.slf4j:nlog4j:1.2.24
org.springframework:spring-aspects:2.0.2
commons-pool:commons-pool:1.4
regexp:regexp:1.3
org.codehaus.cargo:cargo-core-container-orion:0.8
org.apache.felix:org.osgi.core:1.4.0
org.apache.ant:ant-testutil:1.7.0
jboss:jboss-system:4.0.2
com.tonicsystems:jarjar:0.6
org.apache.geronimo.specs:geronimo-j2ee-deployment_1.1_spec:1.0
org.hibernate:hibernate-entitymanager:3.3.2.GA
org.springframework:spring-core:2.0.2
commons-beanutils:commons-beanutils:1.8.3
slide:webdavlib:2.0
javacc:javacc:3.2
org.apache.poi:poi:3.0.1-FINAL
org.fusesource.jansi:jansi:1.6
com.sun.jmx:jmxri:1.2.1
jdbm:jdbm:1.0
org.apache.felix:org.apache.felix.main:2.0.2
javax.resource:connector:1.0
cglib:cglib-nodep:2.2
jboss:jboss-j2se:200504122039
emma:emma:2.1.5320
org.testng:testng:6.5.2
org.bouncycastle:bcprov-jdk14:1.45
org.codehaus.janino:commons-compiler:2.6.1
org.apache.jackrabbit:jackrabbit-jcr-commons:1.5.2
javax.sql:jdbc-stdext:2.0
org.codehaus.cargo:cargo-core-api-util:0.8
commons-attributes:commons-attributes-api:2.2
opensymphony:oscache:2.1
org.apache.ant:ant-nodeps:1.7.1
org.apache.commons:commons-compress:1.1
javax.security:jacc:1.0
antlr:antlr:2.7.2
rhino:js:1.6R2
commons-jxpath:commons-jxpath:1.3
${pom.groupId}:org.apache.felix.shell:1.4.1
hibernate:antlr:2.7.5H3
org.springframework:spring-dao:2.0.2
aspectj:aspectjweaver:1.5.3
htmlunit:htmlunit:1.8
com.martiansoftware:jsap:2.1
com.icegreen:greenmail:1.3
tomcat:naming-java:5.0.28
httpunit:httpunit:1.6.1
org.codehaus.jcsp:jcsp:1.1-rc5
org.springframework:spring-aop:2.0.2
javax.xml.soap:saaj-api:1.3
bouncycastle:bcmail-jdk14:138
xom:xom:1.0b3
org.apache.felix:org.osgi.compendium:1.4.0
servletapi:servletapi:2.3
org.freemarker:freemarker:2.3.14
org.apache.xmlgraphics:batik-anim:1.7
groovy:groovy:1.0
org.codehaus.groovy:groovy:1.5.6
proxool:proxool:0.8.3
commons-primitives:commons-primitives:1.0
jboss:javassist:3.3.ga
com.caucho:burlap:2.1.12
ant:ant:1.5.1
org.codehaus.cargo:cargo-core-api-module:0.8
org.subethamail:subethasmtp:2.1.0
xpp3:xpp3:1.1.3.3
radeox:radeox-oro:0.9
opensymphony:quartz-all:1.6.0
org.apache.commons:commons-vfs2:2.0
commons-validator:commons-validator:1.3.1
org.springframework:spring-beans:2.0.2
org.codehaus.cargo:cargo-core-container-geronimo:0.8
jaxen:jaxen:1.1-beta-6
org.apache.bsf:bsf-api:3.1
org.codehaus.jsr166-mirror:jsr166y:1.7.0
commons-collections:commons-collections:20040616
commons-jelly:commons-jelly-tags-xml:1.0
commons-jelly:commons-jelly-tags-log:1.0
com.google.inject:guice:2.0
net.sourceforge.jexcelapi:jxl:2.6
org.slf4j:integration:1.7.6
org.codehaus.cargo:cargo-ant:0.8
com.mockrunner:mockrunner-jdk1.3-j2ee1.3:0.4
javax.resource:connector-api:1.5
xalan:xalan:2.5.1
org.codehaus.gpars:gpars:1.0.0
${pom.groupId}:org.apache.felix.shell.tui:1.4.1
org.codehaus.cargo:cargo-core-uberjar:0.8
jtidy:jtidy:4aug2000r7-dev
velocity:velocity-dep:1.4
org.jdom:jaxen-jdom:1.0-FCS
org.apache.mina:mina-integration-jmx:1.1.7
com.servlets:cos:05Nov2002
commons-configuration:commons-configuration:1.8
commons-modeler:commons-modeler:2.0
org.codehaus.cargo:cargo-core-api-generic:0.8
com.google.guava:guava:18.0
log4j:log4j:1.2.17
junitperf:junitperf:1.8
xerces:xerces-impl:2.6.2
org.apache.shale:shale-test:1.0.4
org.springframework:spring-jpa:2.0.2
org.apache.xmlgraphics:batik-bridge:1.7
geronimo-spec:geronimo-spec-jta:1.0.1B-rc2
com.jcraft:jsch:0.1.42
jasperreports:jasperreports:2.0.5
org.codehaus.cargo:cargo-core-container-tomcat:0.8
${pom.groupId}:org.apache.felix.bundlerepository:1.4.2
ch.qos.logback:logback-core:1.1.2
logkit:logkit:1.0.1
commons-jelly:commons-jelly-tags-junit:1.0
com.oracle:oc4j:1.0
stax:stax-ri:1.0
jotm:jotm_iiop_stubs:2.0.10
xmlunit:xmlunit:1.1
net.sf.kxml:kxml2:2.3.0
org.slf4j:jul-to-slf4j:1.7.6
org.apache.ibatis:ibatis-sqlmap:2.3.4.726
org.apache.geronimo.specs:geronimo-ejb_2.1_spec:1.0
org.hamcrest:hamcrest-core:1.3
org.apache.ant:ant-antlr:1.8.4
com.cenqua.clover:clover:1.3.13
org.apache.jackrabbit:jackrabbit-webdav:1.5.2
commons-attributes:commons-attributes-compiler:2.2
org.springframework:spring-remoting:1.2.8
javax.jms:jms:1.1
com.sun.jdmk:jmxtools:1.2.1
jgroups:jgroups-all:2.4.1
org.apache.xmlgraphics:batik-css:1.7
asm:asm-commons:2.2.3
com.bea.xml:jsr173-ri:1.0
org.apache.felix:org.osgi.foundation:1.2.0
c3p0:c3p0:0.9.1.2
jboss:jboss-archive-browsing:5.0.0alpha-200607201-119
woodstox:wstx-asl:3.2.2
org.apache.directory.server:apacheds-core:1.0-RC3
com.h2database:h2:1.2.132
jdom:jdom:1.0
com.lowagie:itext:2.0.7
tomcat:catalina:5.5.23
jmock:jmock:1.0.0
com.bea.wlplatform:commonj-twm:1.1
org.acegisecurity:acegi-security:1.0.3
stax:stax:1.2.0
org.apache.xmlgraphics:batik-xml:1.7
org.codehaus.cargo:cargo-core-container-jboss:0.8
com.google.code.findbugs:jsr305:1.3.9
groovy:groovy-all-minimal:1.0
org.apache.geronimo.specs:geronimo-servlet_2.4_spec:1.0
cactus:cactus:12-1.4.1
org.beanshell:bsh:2.0b4
org.codehaus.cargo:cargo-core-container-weblogic:0.8
joda-time:joda-time:1.6
org.hibernate:ejb3-persistence:1.0.1.GA
org.springframework:spring-context:2.0.2
commons-javaflow:commons-javaflow:20060411
org.codehaus.cargo:cargo-core-container-resin:0.8
struts:struts:1.2.9
org.slf4j:slf4j-ext:1.7.6
mockobjects:mockobjects-core:0.09
aspectj:aspectjrt:1.5.3
junit:junit:4.12-beta-1
xpp3:xpp3_min:1.1.4c
nekohtml:nekohtml:0.9.5
pull-parser:pull-parser:2
org.slf4j:slf4j-log4j12:1.7.6
com.thoughtworks.qdox:qdox:1.12
com.google.protobuf:protobuf-java:2.4.1
org.springframework:spring-support:1.2.8
org.mockito:mockito-core:1.9.0
org.apache.commons:commons-math3:3.3
com.oracle:toplink-essentials:2.41
org.codehaus.groovy:groovy-all:2.0.7
org.easymock:easymock:3.1
org.objenesis:objenesis:1.0
quartz:quartz:1.6.0
org.codehaus.cargo:cargo-core-api-container:0.8
xml-apis:xml-apis-ext:1.3.04
org.springframework:spring-jdbc:2.0.2
net.sf.hibernate:hibernate:2.1.8
saxpath:saxpath:1.0-FCS
javax.servlet:servlet-api:2.3
xml-resolver:xml-resolver:1.2
org.springframework:spring-agent:2.0.2
javax.jcr:jcr:1.0
commons-codec:commons-codec:1.5
org.codehaus.woodstox:wstx-asl:3.2.7
mockobjects:mockobjects-jdk1.4-j2ee1.3:0.09
org.apache.xmlgraphics:batik-util:1.7
javassist:javassist:3.4.GA
com.experlog:xapool:1.5.0
org.easymock:easymockclassextension:3.1
idb:idb:3.26
org.springframework:spring-mock:2.0.2
forehead:forehead:1.0-beta-5
commons-digester:commons-digester:1.8.1
xerces:xerces:2.4.0
com.oracle.toplink:toplink:10.1.3
org.apache.ant:ant-junit:1.8.4
javax.faces:jsf-api:1.1
org.jdom:jdom:1.1
jboss:jboss-jee:4.2.0.GA
xml-apis:xml-apis:1.0.b2
org.apache.geronimo.specs:geronimo-jms_1.1_spec:1.0
commons-dbcp:commons-dbcp:1.2.2
velocity:velocity:1.5
gsbase:gsbase:2.0.1
commons-jelly:commons-jelly:1.0
org.apache.commons:commons-jexl:2.1.1
openejb:openejb-loader:1.0
itext:itext:1.3
ch.qos.cal10n:cal10n-api:0.8.1
org.yaml:snakeyaml:1.6
asm:asm-tree:2.2.3
commons-logging:commons-logging:1.1.1
bouncycastle:bcprov-jdk14:138
org.apache.mina:mina-core:
jruby:jruby:0.9.2
commons-fileupload:commons-fileupload:1.2
jotm:jotm_jrmp_stubs:2.0.10
org.apache.xmlgraphics:batik-parser:1.7
org.apache.velocity:velocity:1.6.2
org.slf4j:log4j-over-slf4j:1.7.6
org.apache.openejb:javaee-api:5.0-2
org.ccil.cowan.tagsoup:tagsoup:0.9.7
swarmcache:swarmcache:1.0RC2
jaxme:jaxme-api:0.3
com.jamonapi:jamon:2.4
org.hibernate:hibernate-annotations:3.3.1.GA
${pom.groupId}:org.apache.felix.framework:2.0.2
jboss:jboss-cache:1.2.2
org.hibernate:hibernate:3.2.6.ga
org.json:json:20080701
org.apache.ant:ant-launcher:1.7.1
xstream:xstream:1.2
ant:ant-junit:1.6.5
org.bouncycastle:bcpg-jdk14:1.45
javax.portlet:portlet-api:1.0
it.unimi.dsi:fastutil:6.5.15
eclipse:jdtcore:[3.1.0,)
jfree:jcommon:[1.0.0,)
ch.qos.logback:logback-classic.jar:2.2.2
org.apache.geronimo.specs:geronimo-jta_1.0.1B_spec:1.0
ant:ant-nodeps:1.6.2
portlet-api:portlet-api:1.0
groovy:groovy-all:1.0-beta-10
openejb:openejb-core:1.0
org.mockejb:mockejb:0.6-beta2
commons-vfs:commons-vfs:1.0
mysql:mysql-connector-java:5.1.9
net.sf.ehcache:ehcache:1.5.0
javax.servlet:jsp-api:2.0
asm:asm-analysis:2.2
ant:ant-trax:1.6.2
com.keypoint:png-encoder:1.5
checkstyle:checkstyle:4.3
org.apache.ant:ant:1.7.1
javax.persistence:persistence-api:1.0
org.codehaus.janino:janino:2.6.1
commons-httpclient:commons-httpclient:2.0.2
mx4j:mx4j-jmx:2.1.1
javax.jdo:jdo2-api:2.0
com.beust:jcommander:1.12
commons-logging:commons-logging-api:1.1
org.apache.ivy:ivy:2.2.0
net.sf.jsr107cache:jsr107cache:1.0
commons-beanutils:commons-beanutils-bean-collections:1.7.0
org.slf4j:slf4j-api:1.7.7
commons-discovery:commons-discovery:20030211.213356
asm:asm-util:2.2.3
ant:ant-launcher:1.6.2
jotm:jotm:2.0.10
freemarker:freemarker:2.3.8
tomcat:naming-common:5.0.28
com.megginson.sax:xml-writer:0.2
org.hibernate:hibernate-commons-annotations:3.0.0.ga
org.multiverse:multiverse-beta:0.7-RC-1
com.jcraft:jzlib:1.0.7
org.apache.xmlgraphics:batik-ext:1.7
org.apache.directory.server:apacheds-core-shared:1.0-RC3
org.apache.tiles:tiles-core:2.0.6
concurrent:concurrent:1.3.4
asm:asm:2.2.3
org.apache.axis:axis:1.4
cglib-nodep:cglib-nodep:2.1_3
javax.j2ee:j2ee:1.4
org.apache.xmlgraphics:batik-svg-dom:1.7
hessian:hessian:3.0.20
jboss:jboss-common:4.0.2
velocity-tools:velocity-tools-generic:1.4
cactus:cactus-ant:1.4.1
org.apache.xmlgraphics:batik-js:1.7
hsqldb:hsqldb:1.8.0.7
commons-jexl:commons-jexl:1.0
cas:casclient:2.0.11

Switching to fastutil in the OPAL plugin

Is your feature request related to a problem? Please describe.

Processing some artifacts with OPAL plugin causes OutOfMemoryError during external cha generating process.

Describe the solution you'd like

Using data structures with a small memory footprint and fast access (e.g. fastutil) might help.

Additional context

No additional context

Dockerize the REST API

It might be possible to use openjdk:11.0.9-jre-slim and simply run the REST API JAR file.
We might force the creation of the JAR file via a:

Maven goal
Docker stage
Making the main Docker-compose service depend on a "builder" service

That's also my personal order of preference for this.

If that's not possible, I'm afraid we would need to use maven images, which I think are even bigger in general.

Update 'repoUrl' field location in wiki

The repoUrl field of the fasten.POMAnalyzer.* Kafka topic has been moved from input to payload in #71.
That wiki paragraph needs to be updated accordingly.

Add 'release time' in the output topic of RepoCloner.

Problem

Version tags may not be always available for the repository, so we will have problem to check out the right version of source code in this case. By adding the 'release time', it will allow some heuristic based checkout of the right version when tags are absent.

Solution

Add 'release time' or any other useful information (not sure yet, if anyone has suggestions) to help checking out the right version of source code in the output topic of RepoCloner.

Include metadata in base entities endpoints

General design question; why do we have different endpoints for {entity} and {entity}/metadata, for example {pkg_ver} and {pkg_ver}/metadata? When retrieving an entity, I would expect all related metadata to be retrieved as well.

Originally posted by @gousiosg in #80 (comment)

Also, the Wiki should be updated accordingly.

Wrong uniqueness constraint for `callables` table

I think the uniqueness constraints for table callables is wrong. Let's take
callable 331721314 (current Java database) and see some info for it:

select p.package_name, pv.version, pv.created_at, m.namespace, c.fasten_uri
from packages p
  join package_versions pv on p.id = pv.package_id
  join modules m on pv.id = m.package_version_id
  join callables c on c.module_id = m.id
where c.id = 331721314;

org.neo4j.neo4j-graph-algo | 3.5.1   | 2018-12-20 13:27:36 | /org.neo4j.graphalgo.impl.shortestpath/Dijkstra$DijkstraIterator | /org.neo4j.graphalgo.impl.shortestpath/Dijkstra$DijkstraIterator.InitQueue()%2Fjava.lang%2FVoidType

This corresponds to the following code: https://github.com/neo4j/neo4j/blob/3.5/community/graph-algo/src/main/java/org/neo4j/graphalgo/impl/shortestpath/Dijkstra.java#L223

Then, we take the edges:

select e.source_id, e.target_id, c.fasten_uri 
from edges e join callables c on c.id = e.target_id
where source_id = 331721314;

 331721314 |  15052438 | /org.neo4j.graphalgo.impl.shortestpath/DijkstraPriorityQueueFibonacciImpl.DijkstraPriorityQueueFibonacciImpl(%2Fjava.util%2FComparator)%2Fjava.lang%2FVoidType
 331721314 |  15052441 | /org.neo4j.graphalgo.impl.shortestpath/DijkstraPriorityQueueFibonacciImpl.insertValue(%2Forg.neo4j.graphdb%2FNode,%2Fjava.lang%2FObject)%2Fjava.lang%2FVoidType
 331721314 |  15052299 | /org.neo4j.graphalgo.impl.shortestpath/DijkstraPriorityQueueImpl.insertValue(%2Forg.neo4j.graphdb%2FNode,%2Fjava.lang%2FObject)%2Fjava.lang%2FVoidType
 331721314 |         1 | ///java.lang/NullPointerException.NullPointerException()VoidType
 331721314 |     16485 | ///java.util/HashMap.put(%2Fjava.lang%2FObject,%2Fjava.lang%2FObject)%2Fjava.lang%2FObject

Node 16485 corresponds to line 227. Node 1 I don't know, but I suspect it is thrown somewhere. Nodes 15052438 is line 225 whereas 15052299 and 15052441 are the result of a virtual dispatch on line 226. This looks good, until...

select p.package_name, pv.version, pv.created_at, m.namespace, c.fasten_uri
from packages p
  join package_versions pv on p.id = pv.package_id
  join modules m on pv.id = m.package_version_id
  join callables c on c.module_id = m.id
where c.id = 15052299;

org.neo4j.neo4j-graph-algo | 3.3.0-alpha06 | 2017-08-17 10:12:43 | /org.neo4j.graphalgo.impl.shortestpath/DijkstraPriorityQueueImpl | /org.neo4j.graphalgo.impl.shortestpath/DijkstraPriorityQueueImpl.insertValue(%2Forg.neo4j.graphdb%2FNode,%2Fjava.lang%2FObject)%2Fjava.lang%2FVoidType

The version of the function the callgraph points to is different (older).

I suspect this is because during upsert we only query the uniquness of a FASTEN URI that does not include a version number. So I 'd like to suggest that we change the uniqueness constraint to include the field module_id, which is already versioned.

Migrate package generation workflow to the new GitHub Container Registry

The generated docker images are being published to docker.pkg.github.com. GitHub deprecated this endpoint. We need to migrate both the image generation actions and the deployment files to the new end point.

Instructions on how to do the migration can be found here

Add type of SCM for checking out source code in the output of RepoCloner

Problem

As there are multiple repo types of SCM (git, hg, svn, ...) used for storing the repository of maven project, it will be good to have information of which type of SCM associated with the 'repoPath'.

Solution

Add 'repoType' field in the fasten.RepoCloner.out

Some metadataDB tests mock their target functions in their entirety

As an example, the following test

fasten/core/src/test/java/eu/fasten/core/data/metadatadb/MetadataDaoTest.java

Lines 89 to 117 in b674bd3

 @Test 

 public void insertPackageTest() { 

 long id = 1; 

 var packageName = "package1"; 

 var forge = "mvn"; 

 var projectName = "project1"; 

 var repository = "repository1"; 

 var createdAt = new Timestamp(1); 

 var insertValues = Mockito.mock(InsertValuesStep5.class); 

 Mockito.when(context.insertInto(Packages.PACKAGES, 

 Packages.PACKAGES.PACKAGE_NAME, Packages.PACKAGES.FORGE, 

 Packages.PACKAGES.PROJECT_NAME, Packages.PACKAGES.REPOSITORY, 

 Packages.PACKAGES.CREATED_AT)).thenReturn(insertValues); 

 Mockito.when(insertValues.values(packageName, forge, projectName, repository, createdAt)).thenReturn(insertValues); 

 var insertOnConflict = Mockito.mock(InsertOnConflictDoUpdateStep.class); 

 Mockito.when(insertValues.onConflictOnConstraint(Keys.UNIQUE_PACKAGE_FORGE)).thenReturn(insertOnConflict); 

 var insertOnDuplicate = Mockito.mock(InsertOnDuplicateSetStep.class); 

 Mockito.when(insertOnConflict.doUpdate()).thenReturn(insertOnDuplicate); 

 var insertOnDuplicateMore = Mockito.mock(InsertOnDuplicateSetMoreStep.class); 

 Mockito.when(insertOnDuplicate.set(Packages.PACKAGES.PROJECT_NAME, Packages.PACKAGES.as("excluded").PROJECT_NAME)).thenReturn(insertOnDuplicateMore); 

 Mockito.when(insertOnDuplicateMore.set(Packages.PACKAGES.REPOSITORY, Packages.PACKAGES.as("excluded").REPOSITORY)).thenReturn(insertOnDuplicateMore); 

 Mockito.when(insertOnDuplicateMore.set(Packages.PACKAGES.CREATED_AT, Packages.PACKAGES.as("excluded").CREATED_AT)).thenReturn(insertOnDuplicateMore); 

 var insertResult = Mockito.mock(InsertResultStep.class); 

 Mockito.when(insertOnDuplicateMore.returning(Packages.PACKAGES.ID)).thenReturn(insertResult); 

 var record = new PackagesRecord(id, packageName, forge, projectName, repository, createdAt); 

 Mockito.when(insertResult.fetchOne()).thenReturn(record); 

 long result = metadataDao.insertPackage(packageName, forge, projectName, repository, createdAt); 

 assertEquals(id, result); 

 }

mocks the entire behavior of its target function:

fasten/core/src/main/java/eu/fasten/core/data/metadatadb/MetadataDao.java

Lines 59 to 82 in b674bd3

 /** 

  * Inserts a record in 'packages' table in the database. 

  * 

  * @param packageName Name of the package 

  * @param forge Forge of the package 

  * @param projectName Project name to which package belongs 

  * @param repository Repository to which package belongs 

  * @param createdAt Timestamp when package was created 

  * @return ID of the new record 

  */ 

 public long insertPackage(String packageName, String forge, String projectName, 

 String repository, Timestamp createdAt) { 

 var resultRecord = context.insertInto(Packages.PACKAGES, 

 Packages.PACKAGES.PACKAGE_NAME, Packages.PACKAGES.FORGE, 

 Packages.PACKAGES.PROJECT_NAME, Packages.PACKAGES.REPOSITORY, 

 Packages.PACKAGES.CREATED_AT) 

 .values(packageName, forge, projectName, repository, createdAt) 

 .onConflictOnConstraint(Keys.UNIQUE_PACKAGE_FORGE).doUpdate() 

 .set(Packages.PACKAGES.PROJECT_NAME, Packages.PACKAGES.as("excluded").PROJECT_NAME) 

 .set(Packages.PACKAGES.REPOSITORY, Packages.PACKAGES.as("excluded").REPOSITORY) 

 .set(Packages.PACKAGES.CREATED_AT, Packages.PACKAGES.as("excluded").CREATED_AT) 

 .returning(Packages.PACKAGES.ID).fetchOne(); 

 return resultRecord.getValue(Packages.PACKAGES.ID); 

 }

After a brief yet super-productive call with @MihhailSokolov we figured out that these tests originally targetted the data access layer, hence mocking the database layer. Due to the lack of the first, a lot of tests ended up targeting an entirely mocked-up version of their corresponding target function.

What I would personally do is to mimic an actual insertion of data in a temporary (test) database and consequently write some assertions relative to it, but this sounds very much like an integration test that might already be present. Either way, I would get rid of these tests.

javagc-wala fails with xwiki-platform-filter-event-model-12.4.jar

Command used: java -jar javacg-wala-0.0.1-SNAPSHOT-with-dependencies.jar -jre /media/DATA/utils/jdk8 -f xwiki-platform-filter-event-model-12.4.jar -o .

The jar can be found on https://nexus.xwiki.org/nexus/content/repositories/public/org/xwiki/platform/xwiki-platform-filter-event-model/12.4/xwiki-platform-filter-event-model-12.4.jar.

Log:

[main] INFO eu.fasten.analyzer.javacgwala.Main - Could not load wala.properties
WARNING: java_runtime_dir /Library/Java/JavaVirtualMachines/jdk1.8.0_241.jdk/Contents/Home in wala.properties is invalid.  Using boot class path instead.
java.lang.IllegalStateException: Could not create a entrypoint callsites: 1. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/management/DynamicMBean
2. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/security/auth/Destroyable
3. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/security/auth/login/Configuration$Parameters
4. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/Data
5. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/NodeSetData
6. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/URIDereferencer
7. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/XMLStructure
8. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dom/DOMURIReference
9. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/CanonicalizationMethod
10. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/DigestMethod
11. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/Manifest
12. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/Reference
13. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/SignatureMethod
14. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/SignatureProperties
15. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/SignatureProperty
16. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/SignedInfo
17. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/Transform
18. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/XMLObject
19. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/XMLSignature
20. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/XMLSignature$SignatureValue
21. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/keyinfo/KeyInfo
22. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/keyinfo/KeyName
23. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/keyinfo/KeyValue
24. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/keyinfo/PGPData
25. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/keyinfo/RetrievalMethod
26. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/keyinfo/X509Data
27. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljavax/xml/crypto/dsig/keyinfo/X509IssuerSerial
28. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaBeansAccess
29. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaIOAccess
30. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaIOFileDescriptorAccess
31. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaIOFilePermissionAccess
32. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaIORandomAccessFileAccess
33. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaLangAccess
34. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaLangInvokeAccess
35. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaLangModuleAccess
36. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaLangRefAccess
37. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaNetHttpCookieAccess
38. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaNetInetAddressAccess
39. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaNetSocketAccess
40. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaNetURLAccess
41. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaNetURLClassLoaderAccess
42. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaNetUriAccess
43. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaNioAccess
44. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaNioAccess$BufferPool
45. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaSecurityAccess
46. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaSecurityAccess$ProtectionDomainCache
47. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaSecuritySignatureAccess
48. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaUtilJarAccess
49. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaUtilResourceBundleAccess
50. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/JavaUtilZipFileAccess
51. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/misc/Signal$Handler
52. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/reflect/LangReflectAccess
53. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/vm/compiler/collections/EconomicMap
54. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/vm/compiler/collections/MapCursor
55. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/vm/compiler/word/Pointer
56. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/vm/compiler/word/SignedWord
57. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/internal/vm/compiler/word/UnsignedWord
58. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/vm/ci/code/ValueKindFactory
59. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/vm/ci/hotspot/HotSpotVMEventListener
60. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/vm/ci/meta/ConstantPool
61. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/vm/ci/meta/InvokeTarget
62. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/vm/ci/meta/JavaKind$FormatWithToString
63. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/vm/ci/meta/JavaMethod
64. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/vm/ci/meta/JavaType
65. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/vm/ci/meta/SerializableConstant
66. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/vm/ci/meta/SpeculationLog$SpeculationReason
67. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Ljdk/vm/ci/runtime/JVMCICompiler
68. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/management/spi/PlatformMBeanProvider$PlatformComponent
69. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/nio/ch/DirectBuffer
70. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/nio/ch/Interruptible
71. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/security/util/MessageDigestSpi2
72. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/util/locale/provider/LocaleServiceProviderPool$LocalizedObjectGetter
73. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/util/logging/internal/LoggingProviderImpl$LogManagerAccess

	at com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.makeCallGraph(PropagationCallGraphBuilder.java:241)
	at eu.fasten.analyzer.javacgwala.data.callgraph.CallGraphConstructor.generateCallGraph(CallGraphConstructor.java:90)
	at eu.fasten.analyzer.javacgwala.data.callgraph.PartialCallGraph.generateERCG(PartialCallGraph.java:144)
	at eu.fasten.analyzer.javacgwala.Main.run(Main.java:162)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1729)
	at picocli.CommandLine.access$900(CommandLine.java:145)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2101)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2068)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:1935)
	at picocli.CommandLine.execute(CommandLine.java:1864)
	at eu.fasten.analyzer.javacgwala.Main.main(Main.java:133)
java.lang.NullPointerException
	at eu.fasten.analyzer.javacgwala.Main.writeCallgraph(Main.java:348)
	at eu.fasten.analyzer.javacgwala.Main.run(Main.java:167)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1729)
	at picocli.CommandLine.access$900(CommandLine.java:145)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2101)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2068)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:1935)
	at picocli.CommandLine.execute(CommandLine.java:1864)
	at eu.fasten.analyzer.javacgwala.Main.main(Main.java:133)

/media/DATA/utils/jdk8 leads to Java 1.8.0_252 but the warning in the log suggest that -jre parameter is not really taken into account and that the current Java version (11.0.7) is used.

I get the same error with various other jars you can find in the attached
result.log file. The jars can all be found in https://nexus.xwiki.org/nexus/content/repositories/public/org/xwiki/platform/ (this log is the result of a script executing javacg-wala on all the xwiki-platform 12.4 jars).

Note that xwiki-platform-filter-event-model-12.4.jar does not manipulate most of the classes listed in the error and I was able to reproduce the same error in empty JAR files too so definitely not related to what the JAR uses but it does work well for many of the xwiki-platform JARs so it does not seems to be an issue with the command line.

Rewrite POM Analyzer to use proper MavenResolver

Is your feature request related to a problem? Please describe.

Currently, everything in POM Analyzer is extracted by manually inspecting the POM files of the artifact and its parents. Since everything is done manually, it might not be as precise and correct as it should be.

Describe the solution you'd like

Using a proper MavenResolver would greatly simplify the code and be much more accurate.

Nested "input" field in Kafka topic 'fasten.RepoCloner.out'

The 'fasten.RepoCloner.out' topic has nested "input" field from its dependended topic 'fasten.POMAnalyser.out'. Should only include contents in the "payload" field from topic 'fasten.POMAnalyser.out'.

{
  "input": {
    "input": {
      "date": 1525962137,
      "groupId": "com.orientechnologies",
      "artifactId": "orientdb-studio",
      "version": "2.2.35"
    },
    "plugin_version": "0.1.0",
    ...
}

REST API unit tests

Endpoints are implemented in the eu.fasten.analyzer.restapiplugin.api.mvn.impl package, where they all use methods of the Metadata Data Access Object MetadataDao.java.

Testing those methods requires DB mocking, and it could be done with Testcontainers as in #92.
I would use Java's Dependency Injection to inject the mock DB during tests. More specifically, I think the following field should be @Injected:

fasten/core/src/main/java/eu/fasten/core/data/metadatadb/MetadataDao.java

Line 43 in c6d8017

private DSLContext context;

Such a field is normally initialized with a setter of the DBConnector interface:

fasten/analyzer/restapi-plugin/src/main/java/eu/fasten/analyzer/restapiplugin/RestAPIPlugin.java

Lines 102 to 106 in c6d8017

 @Override 

 public void setDBConnection(DSLContext dslContext) { 

 kbDslContext = dslContext; 

 kbDao = new MetadataDao(kbDslContext); 

 }

But we do not want to start the whole extension (hence the whole REST server) to be able to run unit tests.

Add FastenServer tests

Is your feature request related to a problem? Please describe.

FastenServer does not have any tests which makes the code more error-prone.

Describe the solution you'd like

Some tests must be written to at least test FastenServer basic functionality like setting database connection for plugins and etc.

Change REST API response format

The output seems to be a JSON dump of a JOOQ record. We need to remove the fields field and replace records with just its contents

There is excessive quoting. For example this "{\"access\": \"protected\", \"defined\": true}" can be simply replaced by "{"access": "protected", "defined": true}

Originally posted by @gousiosg in #80 (comment)

Query different metadata in package versions endpoint

Moreover, the query that is used to serve the versions endpoint is wrong

 select "public"."package_versions"."id", "public"."package_versions"."package_id", "public"."package_versions"."version", "public"."package_versions"."cg_generator", "public"."package_versions"."created_at", "public"."package_versions"."metadata", "public"."callables"."metadata" as "callable_metadata" from "public"."packages" join "public"."package_versions" on "public"."packages"."id" = "public"."package_versions"."package_id" join "public"."modules" on "public"."package_versions"."id" = "public"."modules"."package_version_id" join "public"."callables" on "public"."modules"."id" = "public"."callables"."module_id" where lower("public"."packages"."package_name") = lower('org.eclipse.jetty:jetty-webapp') limit 10

Why do you need to join with callables and modules to retrieve the versions?

Originally posted by @gousiosg in #80 (comment)

[...]

From version metadata. Vulnerabilities will be written first there and propagated to the appropriate callables

Originally posted by @gousiosg in #80 (comment)

Copy the "repoUrl" field in Kafka topic 'fasten..RepoCloner.out'

Is your feature request related to a problem? Please describe.

The FASTEN QMSTR plugin needs the repository URL in order for the license and compliance analysis process to begin.
Currently, the repoUrl field is available in fasten.POMAnalyzer.out under input: this represents a problem as the order of nested inputs can change depending on the order of plugins in the pipeline.

Describe the solution you'd like

As suggested in #48 (comment), the repoUrl field could be copied by the RepoCloner to the fasten.RepoCloner.out topic, under payload.

Additional context

Needed by #48.

	@Test
	public void insertPackageTest() {
	long id = 1;
	var packageName = "package1";
	var forge = "mvn";
	var projectName = "project1";
	var repository = "repository1";
	var createdAt = new Timestamp(1);
	var insertValues = Mockito.mock(InsertValuesStep5.class);
	Mockito.when(context.insertInto(Packages.PACKAGES,
	Packages.PACKAGES.PACKAGE_NAME, Packages.PACKAGES.FORGE,
	Packages.PACKAGES.PROJECT_NAME, Packages.PACKAGES.REPOSITORY,
	Packages.PACKAGES.CREATED_AT)).thenReturn(insertValues);
	Mockito.when(insertValues.values(packageName, forge, projectName, repository, createdAt)).thenReturn(insertValues);
	var insertOnConflict = Mockito.mock(InsertOnConflictDoUpdateStep.class);
	Mockito.when(insertValues.onConflictOnConstraint(Keys.UNIQUE_PACKAGE_FORGE)).thenReturn(insertOnConflict);
	var insertOnDuplicate = Mockito.mock(InsertOnDuplicateSetStep.class);
	Mockito.when(insertOnConflict.doUpdate()).thenReturn(insertOnDuplicate);
	var insertOnDuplicateMore = Mockito.mock(InsertOnDuplicateSetMoreStep.class);
	Mockito.when(insertOnDuplicate.set(Packages.PACKAGES.PROJECT_NAME, Packages.PACKAGES.as("excluded").PROJECT_NAME)).thenReturn(insertOnDuplicateMore);
	Mockito.when(insertOnDuplicateMore.set(Packages.PACKAGES.REPOSITORY, Packages.PACKAGES.as("excluded").REPOSITORY)).thenReturn(insertOnDuplicateMore);
	Mockito.when(insertOnDuplicateMore.set(Packages.PACKAGES.CREATED_AT, Packages.PACKAGES.as("excluded").CREATED_AT)).thenReturn(insertOnDuplicateMore);
	var insertResult = Mockito.mock(InsertResultStep.class);
	Mockito.when(insertOnDuplicateMore.returning(Packages.PACKAGES.ID)).thenReturn(insertResult);
	var record = new PackagesRecord(id, packageName, forge, projectName, repository, createdAt);
	Mockito.when(insertResult.fetchOne()).thenReturn(record);
	long result = metadataDao.insertPackage(packageName, forge, projectName, repository, createdAt);
	assertEquals(id, result);
	}

	/**
	* Inserts a record in 'packages' table in the database.
	*
	* @param packageName Name of the package
	* @param forge Forge of the package
	* @param projectName Project name to which package belongs
	* @param repository Repository to which package belongs
	* @param createdAt Timestamp when package was created
	* @return ID of the new record
	*/
	public long insertPackage(String packageName, String forge, String projectName,
	String repository, Timestamp createdAt) {
	var resultRecord = context.insertInto(Packages.PACKAGES,
	Packages.PACKAGES.PACKAGE_NAME, Packages.PACKAGES.FORGE,
	Packages.PACKAGES.PROJECT_NAME, Packages.PACKAGES.REPOSITORY,
	Packages.PACKAGES.CREATED_AT)
	.values(packageName, forge, projectName, repository, createdAt)
	.onConflictOnConstraint(Keys.UNIQUE_PACKAGE_FORGE).doUpdate()
	.set(Packages.PACKAGES.PROJECT_NAME, Packages.PACKAGES.as("excluded").PROJECT_NAME)
	.set(Packages.PACKAGES.REPOSITORY, Packages.PACKAGES.as("excluded").REPOSITORY)
	.set(Packages.PACKAGES.CREATED_AT, Packages.PACKAGES.as("excluded").CREATED_AT)
	.returning(Packages.PACKAGES.ID).fetchOne();
	return resultRecord.getValue(Packages.PACKAGES.ID);
	}

	@Override
	public void setDBConnection(DSLContext dslContext) {
	kbDslContext = dslContext;
	kbDao = new MetadataDao(kbDslContext);
	}

fasten-project / fasten Goto Github PK

fasten's People

Contributors

Stargazers

Watchers

Forkers

fasten's Issues

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Reproduce the issue

Describe the bug

To Reproduce

Expected behavior

Logs

Environment (please complete the following information):

Additional context

Problem

Solution

Describe the bug

To Reproduce

Expected behavior

Logs

Environment

Is your feature request related to a problem? Please describe.

The packages table

The package_versions table

Inner join between packages and package_versions

The modules table

Inner join between 3. and modules

The callables table

Inner join between 5. and callables

Describe the solution you'd like

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

Problem

Proposed solution

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

Problem

Solution

Problem

Solution

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

Recommend Projects

Recommend Topics

Recommend Org

The `packages` table

The `package_versions` table

Inner join between `packages` and `package_versions`

The `modules` table

Inner join between 3. and `modules`

The `callables` table

Inner join between 5. and `callables`