aperepel / ambari-nifi-service Goto Github PK

Ambari service for easily installing and managing NiFi on HDP cluster

Features:

• By default, downloads a prebuilt version of 0.3.0 (to get around the Solr issue)
• But also gives option to build Nifi from source instead
• Exposes nifi.properties in Ambari UI (so you can configure port etc)
• Runs the service as nifi user

Author: Ali Bajwa

• Download HDP 2.3 sandbox VM image (Sandbox_HDP_2.3_VMware.ova) from Hortonworks website
• Import Sandbox_HDP_2.3_VMware.ova into VMWare and set the VM memory size to 8GB
• Now start the VM
• After it boots up, find the IP address of the VM and add an entry into your machines hosts file e.g.

192.168.191.241 sandbox.hortonworks.com sandbox    

• Connect to the VM via SSH (password hadoop) and start Ambari server

ssh [email protected]
/root/start_ambari.sh

• To download the NiFi service folder, run below

VERSION=`hdp-select status hadoop-client | sed 's/hadoop-client - \([0-9]\.[0-9]\).*/\1/'`
sudo git clone https://github.com/abajwa-hw/ambari-nifi-service.git   /var/lib/ambari-server/resources/stacks/HDP/$VERSION/services/NIFI   

• Restart Ambari

#sandbox
service ambari restart

#non sandbox
sudo service ambari-server restart

• Then you can click on 'Add Service' from the 'Actions' dropdown menu in the bottom left of the Ambari dashboard:

On bottom left -> Actions -> Add service -> check NiFi server -> Next -> Next -> Change any config you like (e.g. port) -> Next -> Deploy

• On successful deployment you will see the NiFi service as part of Ambari stack and will be able to start/stop the service from here:

• When you've completed the install process, NiFi server will appear in Ambari

• You can see the parameters you configured under 'Configs' tab

• One benefit to wrapping the component in Ambari service is that you can now monitor/manage this service remotely via REST API

export SERVICE=NIFI
export PASSWORD=admin
export AMBARI_HOST=localhost
export CLUSTER=Sandbox

#get service status
curl -u admin:$PASSWORD -i -H 'X-Requested-By: ambari' -X GET http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER/services/$SERVICE

#start service
curl -u admin:$PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Start $SERVICE via REST"}, "Body": {"ServiceInfo": {"state": "STARTED"}}}' http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER/services/$SERVICE

#stop service
curl -u admin:$PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Stop $SERVICE via REST"}, "Body": {"ServiceInfo": {"state": "INSTALLED"}}}' http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER/services/$SERVICE

• ...and also install via Blueprint. See example here on how to deploy custom services via Blueprints

• The NiFi webUI login page should come up at the below link: http://sandbox.hortonworks.com:9090/nifi

• You can also open it from within Ambari via iFrame view

• To remove the Nifi service:

  • Stop the service via Ambari

  • Unregister the service

export SERVICE=NIFI export PASSWORD=admin export AMBARI_HOST=localhost export CLUSTER=Sandbox
curl -u admin:$PASSWORD -i -H 'X-Requested-By: ambari' -X DELETE http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER/services/$SERVICE

#if above errors out, run below first to fully stop the service #curl -u admin:$PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Stop $SERVICE via REST"}, "Body": {"ServiceInfo": {"state": "INSTALLED"}}}' http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER/services/$SERVICE ```

• Clear LDAP dir to reset the data in LDAP rm -rf /opt/nifi* rm /tmp/nifi-0.3.0-SNAPSHOT-bin.zip