Simple examples of Spark SQL user-defined functions. Tested with CDH 5.13.1, Spark 2.1.0 (see the spark1.6 branch for Spark 1.x examples)
hdfs dfs -put data/temperatures.json temperatures.json
hdfs dfs -put data/inventory.json inventory.json
Under each example root (java-udf/, scala-udf/, ...):
mvn package
Python UDF:
spark2-submit --master local python-udf-example.py
Scala UDF:
spark2-submit --class com.cloudera.fce.curtis.sparkudfexamples.scalaudf.ScalaUDFExample --master local target/scalaudf-0.0.1-jar-with-dependencies.jar
Java UDF:
spark2-submit --class com.cloudera.fce.curtis.sparkudfexamples.javaudf.JavaUDFExample --master local target/javaudf-0.0.1-jar-with-dependencies.jar
Scala UDAF:
spark2-submit --class com.cloudera.fce.curtis.sparkudfexamples.scalaudaf.ScalaUDAFExample --master local target/scalaudaf-0.0.1-jar-with-dependencies.jar
Hive UDF:
spark2-submit --jars target/hiveudf-0.0.1-jar-with-dependencies.jar hive-udf-example.py
Scala UDAF From PySpark:
spark2-submit --jars target/scalaudaffrompython-0.0.1.jar --driver-class-path target/scalaudaffrompython-0.0.1.jar scala-udaf-from-python.py
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent