To download and extract the installation files for the first time, just run this command :
ZIP=~/setup.zip; cd ~; wget -O $ZIP https://raw.githubusercontent.com/leogouttefarde/smack/master/setup.zip; sudo apt -y install unzip; unzip -o $ZIP
To check for new installation updates, run ~/scripts/update_setup.sh
Run ~/scripts/install_all.sh
Run ~/scripts/uninstall_all.sh
Run ~/scripts/version.sh
Run ~/scripts/run_mesos_cluster.sh
The number of kafka brokers is configured through the script argument
Run ~/scripts/run_kafka_cluster.sh <NB_KAFKA_BROKERS>
Run ~/scripts/run_cassandra_cluster.sh &
Run ~/scripts/stop_mesos_cluster.sh
Run ~/scripts/stop_kafka_cluster.sh
Run ~/scripts/stop_cassandra_cluster.sh
Run ~/scripts/kill_mesos_server.sh <server>
Run ~/scripts/kill_cassandra_server.sh <server>
Run ~/scripts/kill_kafka_broker.sh <brokerNumber>
wget -O ~/app_demo.zip https://github.com/leogouttefarde/smack/blob/master/app_demo.zip?raw=true; unzip -o app_demo.zip; cd ~/app_demo; sudo ../spark-2.0.2-bin-hadoop2.7/bin/spark-submit --packages datastax:spark-cassandra-connector:2.0.0-M2-s_2.11 --class "SimpleApp" --master mesos://zk://server-1:2181/mesos target/scala-2.11/simple-project_2.11-1.0.jar
First, run ~/scripts/download_demo.sh
to download the application.
scala -classpath "lib/kafka_2.11-0.8.2.0.jar:lib/kafka-clients-0.8.2.0.jar:lib/scala-library-2.10.4.jar:lib/slf4j-api-1.7.2.jar:target/scala-2.10/spark-example-kafka_2.10-1.0.jar" ProducerTest
sudo ../spark-2.0.2-bin-hadoop2.7/bin/spark-submit --packages datastax:spark-cassandra-connector:2.0.0-M2-s_2.11 --jars lib/kafka_2.11-0.10.1.0.jar,lib/kafka-clients-0.10.1.0.jar,lib/slf4j-api-1.7.2.jar --class "ConsumerToSpark" target/scala-2.11/spark-example-kafka_2.11-1.0.jar
cd ~; cp ~/app_demoV2/target/scala-2.11/simple-project_2.11-1.0.jar .; sudo ~/spark-2.0.2-bin-hadoop2.7/bin/spark-submit --packages datastax:spark-cassandra-connector:2.0.0-M2-s_2.11 --class "batch.SelectJob" simple-project_2.11-1.0.jar