Giter VIP home page Giter VIP logo

Comments (8)

florianleibert avatar florianleibert commented on May 19, 2024

You should make sure that the java process is able to load the libmesos.so. You can force it by setting "JAVA_LIBRARY_PATH=/your/directory/where/libmesos.so/is/in". That should fix your issue!

from chronos.

 avatar commented on May 19, 2024

I'm actually seeing this same issue too...

Step 1: Find location of libmesos.so

root@ip-10-1-0-245:/opt/mesos/mesos/third_party/zookeeper-3.3.4# find / | grep libmesos.so
/opt/mesos/lib/libmesos.so
/opt/mesos/src/.libs/libmesos.so
/usr/lib/libmesos.so
/usr/local/lib/libmesos.so

Step 2: Create Startup script and pass JAVA_LIBRARY_PATH (you can see this is being passed int he final line... I also tried setting it using export to no avail):

root@ip-10-1-0-245:/opt/mesos/mesos/third_party/zookeeper-3.3.4# cat /opt/chronos/start.sh 
#JAVA_LIBRARY_PATH=/opt/mesos/lib
cd /opt/mesos/mesos/third_party/zookeeper-3.3.4
./bin/zkServer.sh stop
./bin/zkServer.sh start
cd /opt/chronos/
java -Djava.library.path="/opt/mesos/lib" -cp target/chronos*.jar com.airbnb.scheduler.Main server config/local_scheduler.yml

Step 3: Run the script (it launches, I'll spare you the entire log)

root@ip-10-1-0-245:/opt/mesos/mesos/third_party/zookeeper-3.3.4# /opt/chronos/start.sh 
JMX enabled by default
Using config: /opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
JMX enabled by default
Using config: /opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
WARN  [2013-08-08 20:26:10,077] com.google.common.cache.CacheBuilder: ignoring weigher specified without maximumWeight
INFO  [2013-08-08 20:26:10,128] com.airbnb.scheduler.Main$: ---------------------
INFO  [2013-08-08 20:26:10,128] com.airbnb.scheduler.Main$: Initializing chronos.
INFO  [2013-08-08 20:26:10,128] com.airbnb.scheduler.Main$: ---------------------
INFO  [2013-08-08 20:26:10,221] com.airbnb.scheduler.config.MainModule: Wiring up the application
INFO  [2013-08-08 20:26:10,579] com.airbnb.scheduler.config.ZookeeperModule: localhost:2181
INFO  [2013-08-08 20:26:10,581] com.airbnb.scheduler.config.ZookeeperModule: Using external zookeeper.
INFO  [2013-08-08 20:26:10,587] com.airbnb.scheduler.config.ZookeeperModule: localhost:2181
INFO  [2013-08-08 20:26:10,611] com.airbnb.scheduler.config.ZookeeperModule: Providing MesosStatePersistenceStore
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:host.name=ip-10-1-0-245.ec2.internal
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.7.0_25
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-7-oracle/jre
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=target/chronos-1.0-SNAPSHOT.jar
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/opt/mesos/lib
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
INFO  [2013-08-08 20:26:10,628] org.apache.zookeeper.ZooKeeper: Client environment:os.version=3.2.0-40-virtual

Step 4: Hit the URL for the WebUI and create a new job. Below is the log from the console

INFO  [2013-08-08 20:28:28,079] com.airbnb.scheduler.mesos.MesosJobFramework: No tasks scheduled! Declining offers
I0808 20:28:28.079659 25193 master.cpp:1469] Processing reply for offer 201308082028-4110418186-49206-25173-3 on slave 201308082028-4110418186-49206-25173-0 (ip-10-1-0-245.ec2.internal) for framework chronos
I0808 20:28:28.079859 25193 master.hpp:314] Removing offer with resources cpus=2; mem=6426; ports=[31000-32000]; disk=3631 on slave 201308082028-4110418186-49206-25173-0
I0808 20:28:28.079968 25191 hierarchical_allocator_process.hpp:495] Framework chronos filtered slave 201308082028-4110418186-49206-25173-0 for 5.00secs
I0808 20:28:29.076807 25192 hierarchical_allocator_process.hpp:597] Performed allocation for 1 slaves in 94.11us
INFO  [2013-08-08 20:28:29,927] com.airbnb.scheduler.api.Iso8601JobResource: Received request for job:ScheduleBasedJob(R/2013-08-08T13:28:24Z/PT24H,zzztest,echo "hello world",PT15M,0,0,,,2,[email protected],,,true,0.0,0,0,false)
WARN  [2013-08-08 20:28:29,944] com.airbnb.scheduler.jobs.JobUtils$: Skipped forward 0 iterations, modified start from 'Thursday, August 8, 2013' to 'Thursday, August 8, 2013
WARN  [2013-08-08 20:28:29,946] com.airbnb.scheduler.graph.JobGraph: Adding vertex:zzztest
WARN  [2013-08-08 20:28:29,947] com.airbnb.scheduler.graph.JobGraph: Current number of vertices:1
INFO  [2013-08-08 20:28:29,947] com.airbnb.scheduler.jobs.JobScheduler: Persisting job:zzztest
INFO  [2013-08-08 20:28:29,948] com.airbnb.scheduler.state.MesosStatePersistenceStore: Persisting job 'zzztest' with data 'ScheduleBasedJob(R/2013-08-08T13:28:24Z/PT24H,zzztest,echo "hello world",PT15M,0,0,,,2,[email protected],,,true,0.0,0,0,false)'
WARN  [2013-08-08 20:28:29,985] com.airbnb.scheduler.api.Iso8601JobResource: Exception while serving request
! java.lang.UnsatisfiedLinkError: org.apache.mesos.state.ZooKeeperState.__fetch(Ljava/lang/String;)J
! at org.apache.mesos.state.ZooKeeperState.__fetch(Native Method) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.apache.mesos.state.ZooKeeperState.fetch(ZooKeeperState.java:77) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.airbnb.scheduler.state.MesosStatePersistenceStore.persistData(MesosStatePersistenceStore.scala:143) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.airbnb.scheduler.state.MesosStatePersistenceStore.persistJob(MesosStatePersistenceStore.scala:64) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.airbnb.scheduler.jobs.JobScheduler$$anonfun$registerJob$3.apply(JobScheduler.scala:173) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.airbnb.scheduler.jobs.JobScheduler$$anonfun$registerJob$3.apply(JobScheduler.scala:169) ~[chronos-1.0-SNAPSHOT.jar:na]
! at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59) ~[chronos-1.0-SNAPSHOT.jar:na]
! at scala.collection.immutable.List.foreach(List.scala:45) ~[chronos-1.0-SNAPSHOT.jar:na]
! at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:30) ~[chronos-1.0-SNAPSHOT.jar:na]
! at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:42) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.airbnb.scheduler.jobs.JobScheduler.registerJob(JobScheduler.scala:168) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.airbnb.scheduler.api.Iso8601JobResource.post(Iso8601JobResource.scala:43) ~[chronos-1.0-SNAPSHOT.jar:na]
! at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_25]
! at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_25]
! at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_25]
! at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_25]
! at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.yammer.metrics.jersey.InstrumentedResourceMethodDispatchProvider$TimedRequestDispatcher.dispatch(InstrumentedResourceMethodDispatchProvider.java:32) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.yammer.dropwizard.jersey.OptionalResourceMethodDispatchAdapter$OptionalRequestDispatcher.dispatch(OptionalResourceMethodDispatchAdapter.java:37) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.server.impl.uri.rules.ResourceObjectRule.accept(ResourceObjectRule.java:100) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716) ~[chronos-1.0-SNAPSHOT.jar:na]
! at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1448) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.airbnb.scheduler.api.RedirectFilter.doFilter(RedirectFilter.scala:20) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.yammer.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:29) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.yammer.metrics.jetty.InstrumentedHandler.handle(InstrumentedHandler.java:200) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:264) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.yammer.dropwizard.jetty.BiDiGzipHandler.handle(BiDiGzipHandler.java:123) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.Server.handle(Server.java:368) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.server.nio.BlockingChannelConnector$BlockingChannelEndPoint.run(BlockingChannelConnector.java:298) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) ~[chronos-1.0-SNAPSHOT.jar:na]
! at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
72.87.249.82 - - [08/Aug/2013:20:28:29 +0000] "POST /scheduler/iso8601 HTTP/1.1" 500 1359 810 810

... and the log from the WebUI:

Error creating job 'zzztest'
Server responded with <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/> <title>Error 500 Internal Server Error</title> </head> <body><h2>HTTP ERROR 500</h2> <p>Problem accessing /scheduler/iso8601. Reason: <pre> Internal Server Error</pre></p><br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> </body> </html> .

Step 5:
You can see that zookeeper is running:

ubuntu@ip-10-1-0-245:~$ ps -ef|grep zoo
root     25154     1  0 20:28 pts/0    00:00:00 java -Dzookeeper.log.dir=. -Dzookeeper.root.logger=INFO,CONSOLE -cp /opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../build/classes:/opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../build/lib/*.jar:/opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../conf: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false org.apache.zookeeper.server.quorum.QuorumPeerMain /opt/mesos/mesos/third_party/zookeeper-3.3.4/bin/../conf/zoo.cfg
ubuntu   25227 24325  0 20:30 pts/1    00:00:00 grep --color=auto zoo
ubuntu@ip-10-1-0-245:~$ echo ruok | nc 127.0.0.1 2181
imok

The job actually persists for the session (and runs). But when chronos is terminated, the job is not persisted on the next restart.

Please advise...

from chronos.

andykram avatar andykram commented on May 19, 2024

@buildscientist: I imagine this is because you're running v0.12.0 of Mesos with latest Chronos. Chronos these days requires v0.14 or later. The exception regarding fetch is because there was a breaking change in the JNI bindings after v0.12.0.

from chronos.

 avatar commented on May 19, 2024

@andykram I'm running mesos v0.12.0. Do you have a link to the recommended build of Mesos? (I'm running on Ubuntu 12.04 LTS). The apache foundation is showing v0.12.0 as latest... (link: http://mesos.apache.org/downloads/)

from chronos.

mrdmnd avatar mrdmnd commented on May 19, 2024

For what it's worth, I am seeing this issue on 0.12.0 as well.

from chronos.

andykram avatar andykram commented on May 19, 2024

https://github.com/apache/mesos/tree/0.13.0-rc5 would probably be sufficient. They don't have an official release greater than v0.12.0 at the moment, those are still WIP.

from chronos.

florianleibert avatar florianleibert commented on May 19, 2024

Mesosphere has some public jars & distributions:

https://s3.amazonaws.com/mesosphere-maven-public/org/apache/mesos/mesos_jdk6/0.14.0/mesos_jdk6-0.14.0.jar

https://s3.amazonaws.com/mesos-pkg/ubuntu/12.04/mesos_0.14.0_amd64.deb
https://s3.amazonaws.com/mesos-pkg/ubuntu/12.10/mesos_0.14.0_amd64.deb

from chronos.

 avatar commented on May 19, 2024

@andykram @mrdmnd I can verify that this now works with Mesos v0.13.0 (https://github.com/apache/mesos/tree/0.13.0-rc5).

I cloned this repo and configured with prefix=/opt/mesos. I didn't opt to have the Chronos installer install mesos in prior attempts (I couldn't get it to work on Ubuntu 12.04)... and after re-installing mesos (v 0.13), there was no need to reconfigure anything.

A few gotchas though with your documentation:

  1. I typically clone my repos in the /tmp/ directory. When I did this, I found that the sample_zoo.cfg has my data directory in /tmp/zookeeper. If you're looking to persist data, persisting it in /tmp is probably a bad idea. Also, adding that "--prefix=/opt/mesos" or something similar to your ./configure command will save you some hassle later.

  2. When launching Zookeeper (zkServer.sh), t's going to look for the zoo.cfg file which wont exist by default. Search your system for sample_zoo.cfg (likely ../conf/sample_zoo.cfg) and copy/rename this to zoo.cfg. Be sure to update the dataDir param to be something other than /tmp). You'll need to create this dir ahead of time, fyi.

Hope this helps those having issues in the future, and thanks for the quick help @andykram, @mrdmnd, and @florianleibert.

-Allan Parsons
viddy.com

from chronos.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.