Giter VIP home page Giter VIP logo

ontolo-ci's Introduction

Welcome to Ontolo-CI

Ontolo-CI is a docker based system that integrates with GitHub to provide a continuos integration system for ontologies. It uses Shape Expressions and test instances in order to validate ontologies.

Although inspired in Travis-CI and many other continuos integration systems, Ontolo-CI is focused on develop continuos integration for ontologies. This is done by means of test instances and Shape Expressions. Furthermore, Ontolo-CI is integrated with GitHub, so you can add it as a check to Pull Requests or Pushes to different branches.

The following diagram illustrates the architecture of Ontolo-CI. As can be seen, it can be deployed as a docker container in any machine. Then it will listen to GitHub Webhooks. Whenever a webhook from github arrives it inmediatelly schedulles a build and after the build is finished Ontolo-CI will notify GitHub and publish the data on its webpage.

Docker Container

Ontolo-CI is shipped as a Docker container so that anyone can deploy its own instance on demand. Inside the Docker container there are several components that work together. The following schema shows them.

  • Ontolo-CI Listener: The listener component receives notifications from GitHub when a Pull Request is started or when commits are pushed. This notifies the scheduller about the new build to perform.

  • Ontolo-CI Hub: It acts as a GitHub API interface client. It allows the system to collect files from GitHub but also to inform about the status of the builds.

  • Ontolo-CI Scheduller: This component receives builds to schedule from the listener, then creates a worker with the build and schedules its execution.

  • Ontolo-CI Worker: Each worker contains a build to execute. A build is a set of tests to execute over an ontology. It only knows how to execute tests when told and who to notificate when finished.

  • Database: When a build is finished by a worker the results of the build are stored in a database. Up to now the results that are being stored are: repo, branch, event, result. Where the result stores not only the results of the test cases, but also the execution time and other metrics.

  • Ontolo-CI API: The API provides an access layer for thrid party services that need to explore the data from an Ontolo-CI instance. It is also used by the web service. It only allows reading data at the time.

  • Ontolo-CI Web: Is a Web interface that stores the results of all executions schedulled, under execution and executed.

Deploy

As previously seen, Ontolo-CI is a docker based system. Therefore the only important requirement is that you have Docker installed and running on your computer. Then, you just need to follow these steps.

  1. Download the latest version of the repository, either as a ZIP file or by cloning the master branch.
git clone https://github.com/weso/ontolo-ci
  1. Then move to the docker folder from inside Ontolo-CI. For that:
cd ontolo-ci/docker
  1. Finally, execute docker-compose up. This will generate the corresponding images, deploy the containers and expose the ports.

Ports Configuration: Ontolo-CI uses port 80 for serving the web UI, port 10010 for its API and port 10011 for listening to GitHub.

Examples

Take a look to ontolo-ci-example

ontolo-ci's People

Contributors

mistermboy avatar thewillyhuman avatar labra avatar dependabot[bot] avatar

Stargazers

.mpe avatar DVious avatar Gustavo Publio avatar Hugo Lebredo avatar Álvaro Baños Gómez avatar Dani Fdez avatar Othmane Bakhtaoui avatar avatar avatar

Watchers

avatar James Cloos avatar avatar avatar avatar

Forkers

alvaromartinezticarum ppmticarum lucia1984 crisantojimenez herculescrue

ontolo-ci's Issues

Persist Installation Id

Once the user logs in to the system and accepts the permissions we need to save the Installation Id

Running es.weso.ontoloci.worker.WorkerTest terminates with errors

When run mvn test from the ontoloci-worker directory this is the result:

java.io.FileNotFoundException: secrets/ocitest.appid (No such file or directory)
	at java.base/java.io.FileInputStream.open0(Native Method)
	at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
	at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
	at java.base/java.util.Scanner.<init>(Scanner.java:639)
	at es.weso.ontoloci.hub.utils.KeyUtils.getSecret(KeyUtils.java:70)
	at es.weso.ontoloci.hub.utils.KeyUtils.getAppId(KeyUtils.java:63)
	at es.weso.ontoloci.hub.utils.KeyUtils.getJWT(KeyUtils.java:38)
	at es.weso.ontoloci.hub.repository.impl.GitHubRepositoryProvider.getInstallationId(GitHubRepositoryProvider.java:163)
	at es.weso.ontoloci.hub.OntolociHubImplementation.addTestsToBuild(OntolociHubImplementation.java:61)
	at es.weso.ontoloci.worker.WorkerTest.setUp(WorkerTest.java:49)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.maven.surefire.junit.PojoTestSet.setUpFixture(PojoTestSet.java:227)
	at org.apache.maven.surefire.junit.PojoTestSet.executeTestMethod(PojoTestSet.java:129)
	at org.apache.maven.surefire.junit.PojoTestSet.executeTestMethods(PojoTestSet.java:102)
	at org.apache.maven.surefire.junit.PojoTestSet.execute(PojoTestSet.java:83)
	at org.apache.maven.surefire.junit.JUnit3Provider.executeTestSet(JUnit3Provider.java:121)
	at org.apache.maven.surefire.junit.JUnit3Provider.invoke(JUnit3Provider.java:98)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
java.io.FileNotFoundException: File 'secrets/server-pkcs8.key' does not exist
	at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:297)
	at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1805)
	at es.weso.ontoloci.hub.utils.KeyUtils.loadPrivateKey(KeyUtils.java:85)
	at es.weso.ontoloci.hub.utils.KeyUtils.getJWT(KeyUtils.java:41)
	at es.weso.ontoloci.hub.repository.impl.GitHubRepositoryProvider.getInstallationId(GitHubRepositoryProvider.java:163)
	at es.weso.ontoloci.hub.OntolociHubImplementation.addTestsToBuild(OntolociHubImplementation.java:61)
	at es.weso.ontoloci.worker.WorkerTest.setUp(WorkerTest.java:49)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.maven.surefire.junit.PojoTestSet.setUpFixture(PojoTestSet.java:227)
	at org.apache.maven.surefire.junit.PojoTestSet.executeTestMethod(PojoTestSet.java:129)
	at org.apache.maven.surefire.junit.PojoTestSet.executeTestMethods(PojoTestSet.java:102)
	at org.apache.maven.surefire.junit.PojoTestSet.execute(PojoTestSet.java:83)
	at org.apache.maven.surefire.junit.JUnit3Provider.executeTestSet(JUnit3Provider.java:121)
	at org.apache.maven.surefire.junit.JUnit3Provider.invoke(JUnit3Provider.java:98)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.524 sec <<< FAILURE!
es.weso.ontoloci.worker.WorkerTest.testCaseResultPassedTest()  Time elapsed: 0.523 sec  <<< FAILURE!
java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.maven.surefire.junit.PojoTestSet.setUpFixture(PojoTestSet.java:227)
	at org.apache.maven.surefire.junit.PojoTestSet.executeTestMethod(PojoTestSet.java:129)
	at org.apache.maven.surefire.junit.PojoTestSet.executeTestMethods(PojoTestSet.java:102)
	at org.apache.maven.surefire.junit.PojoTestSet.execute(PojoTestSet.java:83)
	at org.apache.maven.surefire.junit.JUnit3Provider.executeTestSet(JUnit3Provider.java:121)
	at org.apache.maven.surefire.junit.JUnit3Provider.invoke(JUnit3Provider.java:98)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.lang.IllegalArgumentException: Key argument cannot be null.
	at io.jsonwebtoken.lang.Assert.notNull(Assert.java:82)
	at io.jsonwebtoken.impl.DefaultJwtBuilder.signWith(DefaultJwtBuilder.java:120)
	at io.jsonwebtoken.impl.DefaultJwtBuilder.signWith(DefaultJwtBuilder.java:147)
	at es.weso.ontoloci.hub.utils.KeyUtils.getJWT(KeyUtils.java:48)
	at es.weso.ontoloci.hub.repository.impl.GitHubRepositoryProvider.getInstallationId(GitHubRepositoryProvider.java:163)
	at es.weso.ontoloci.hub.OntolociHubImplementation.addTestsToBuild(OntolociHubImplementation.java:61)
	at es.weso.ontoloci.worker.WorkerTest.setUp(WorkerTest.java:49)
	... 19 more
Caused by: java.lang.IllegalArgumentException: Key argument cannot be null.
	at io.jsonwebtoken.lang.Assert.notNull(Assert.java:82)
	at io.jsonwebtoken.impl.DefaultJwtBuilder.signWith(DefaultJwtBuilder.java:120)
	at io.jsonwebtoken.impl.DefaultJwtBuilder.signWith(DefaultJwtBuilder.java:147)
	at es.weso.ontoloci.hub.utils.KeyUtils.getJWT(KeyUtils.java:48)
	at es.weso.ontoloci.hub.repository.impl.GitHubRepositoryProvider.getInstallationId(GitHubRepositoryProvider.java:163)
	at es.weso.ontoloci.hub.OntolociHubImplementation.addTestsToBuild(OntolociHubImplementation.java:61)
	at es.weso.ontoloci.worker.WorkerTest.setUp(WorkerTest.java:49)
	... 19 more


Results :

Failed tests:   es.weso.ontoloci.worker.WorkerTest.testCaseResultPassedTest()

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.930 s
[INFO] Finished at: 2020-12-16T09:10:50Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project worker: There are test failures.
[ERROR] 
[ERROR] Please refer to /ontolo-ci/ontoloci-worker/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Can you take a look to it @mistermboy ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.