Giter VIP home page Giter VIP logo

bigdata-in-practice's Introduction

BigData-In-Practice

大数据项目仓库、涉及 Hadoop、Spark、Kafka、Hbase..... 等，更新中...

综合实践项目

项目名 说明
ImoocLogAnalysis 使用 Spark SQL imooc 访问日志，数据清洗，统计，可视化

入门学习示例

项目名 所属组件 介绍
ch2noaa MapReduce MapReduce 实验 - 计算气温 最大/最小/平均 值
hdfscrud HDFS HDFS Java API 增删查改
hdfslogcollect HDFS Timer 定时将日志文件备份到 HDFS 中去，copyFromLocalFile
commonfans MapReduce MapReduce 计算共同好友
hadoopjoin MapReduce MapReduce 两表进行左连接 left join，两表进行 map join
customizePartition MapReduce MapReduce 自定义分区 ，Partitioner 决定每条记录应该送往哪个reducer节点
sparkSqlSample Spark SQL Spark SQL 样例，关于HiveContext、SQLContext、SparkSession、RDD、DataFrame、Dataset的使用
curator-example Zookeeper 基于Apache Curator实现对Zookeeper的操作，以及数据发布/订阅、负载均衡、命名服务、分布式协调/通知、集群管理、Master选举、分布式锁和分布式队列等Zookeeper的应用场景
HbaseExamples HBase Hbase Java API的基本操作，包括增删查改、过滤器、协处理器，Phoenix、Phoenix+Mybatis等
kafka-example Kafka 深入理解Kafka各种操作，生产者、消费者、主题、分区、应用、可靠性、spark

数据算法 Hadoop + Spark 实现

阅读资料

Canal

bigdata-in-practice's People

Contributors

whirlys avatar

Watchers

avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.