org.apache.mahout.clustering.kmeans.GenKMeansDataset not running. about hibench HOT 10 CLOSED

The script should be ran under the HiBench root directory.

from hibench.

I apologize, I don't seem to understand. I ran it under /root/hibench/kmeans/bin/ directory where the prepare.sh script is.
When I do a "jar -tf datatools.jar" it doesn't show me any of the uncommons packages. Which makes me wonder if the build.xml is not right, but then it seems to include *.jar under the lib directory on the classpath. So I am not really sure?

from hibench.

Which distribution of hadoop did you use?

from hibench.

Cdh4

from hibench.

Can you paste your last line of prepare.sh here?

from hibench.

Sure. Thanks for looking into this.

hadoop jar ${DATATOOLS} org.apache.mahout.clustering.kmeans.GenKMeansDataset -libjars $MAHOUT_HOME/examples/target/mahout-examples-0.7-job.jar ${OPTION}

from hibench.

Here are the last two lines of the latest version of the script. As you can see, the first line make sure the client machine get access to the right jars, the second line make sure the mahout-examples-0.7-job.jar (which include the definition of MersenneTwisterRNG) is distributed to cluster nodes. I can not reproduce the problem here, can you give the full log?

export HADOOP_CLASSPATH=${MAHOUT_HOME}/bin/mahout classpath | tail -1

exec "$HADOOP_EXECUTABLE" --config $HADOOP_CONF_DIR jar ${DATATOOLS} org.apache.mahout.clustering.kmeans.GenKMeansDataset -libjars $MAHOUT_HOME/examples/target/mahout-examples-0.7-job.jar ${COMPRESS_OPT} ${OPTION}

from hibench.

I was able to make it run after I used the mahout classpath command. Somehow I missed it first time around. Thanks for the help.

from hibench.

vinayakponangi, can you please provide more details on the solution?

In case Hadoop is deployed with Cloudera Manager, Ambari etc. one should check if the JAVA_HOME was set properly. It is required for "${MAHOUT_HOME}/bin/mahout classpath"

from hibench.

vinayakponangi, could you provide more details about the solution?

from hibench.