Giter VIP home page Giter VIP logo

spark-submit-ui's Introduction

spark-submit-ui

这是一个基于playframwork开发,web管理的spark应用程序

你需要安装SBT和Java以及PlayFramowrk。项目基于2.2.x 版本开发,需要PlayFramowrk 2.2或更高版本。

测试环境

  • JDK8
  • Center OS 6.5
  • Spark 1.5.2
  • Hadoop 2.6.0
  • Scala 2.11

主要功能

  • hadoop metrics 数据监控
  • spark 集群状态信息展示
  • 完善的spark app 提交与管理
  • 任务状态监控,状态推送

并下载并安装Play Framework 编译环境

Installing Play

修改配置文件,将集群地址替换为你的

文件路径在

conf/web-site.conf

编译与运行

然后去 http://localhost:9000 查看正在运行的服务器。

如果运行有这个界面提示,点击Apply this script now 初始化数据表

编译与运行

 activator run 

项目默认使用H2数据库

这是Play 内嵌的一个数据库 H2 H2官方介绍 http://www.h2database.com/html/main.html

如果想要换成Mysql或者是其他的存储可以参考指引 MySQL 数据库引擎连接属性 配置文件 conf/application.conf

 
db.default.driver=com.mysql.jdbc.Driver
db.default.url="jdbc:mysql://localhost/playdb"
db.default.user=playdbuser
db.default.pass="a strong password" 

#其他

通过界面管理,kill或者rerun任务

如果你的提交参数或配置导致异常,可以在提交时查看相关的错误输出

#Link: IntelliJ IDE支持Maven的和为Scala提供了插件开发.
IntelliJ download: https://www.jetbrains.com/idea/
IntelliJ Scala Plugin: http://plugins.jetbrains.com/plugin/?id=1347
playframework:https://www.playframework.com
sbt:http://www.scala-sbt.org
spark:http://spark.apache.org  

spark-submit-ui's People

Contributors

kingekinge avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.