Topic: spark-streaming Goto Github

Some thing interesting about spark-streaming

👇 Here are 1018 public repositories matching this topic...

abulbasar / pyspark-examples

spark-streaming,Code examples on Apache Spark using python

User: abulbasar

apache spark spark-streaming python

airbnb / airbnb-spark-thrift

spark-streaming,A library for loadling Thrift data into Spark SQL

Organization: airbnb

spark spark-streaming spark-sql thrift

angel-ml / angel

spark-streaming,A Flexible and Powerful Parameter Server for large-scale machine learning

Organization: angel-ml

machine-learning parameter-server spark scala model high-dimensional online-learning spark-streaming

awantik / pyspark-learning

spark-streaming,Updated repository

User: awantik

Home Page: https://github.com/edyoda/pyspark-tutorial

spark spark-streaming pyspark

azure / azure-event-hubs-spark

spark-streaming,Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Organization: azure

spark spark-streaming azure scala eventhubs real-time streaming continuous apache apache-spark

bluishglc / bdp

spark-streaming,A prototype project of big data platform, the source codes of the book Big Data Platform Architecture and Prototype

User: bluishglc

Home Page: https://laurence.blog.csdn.net/article/details/106851739

bigdata prototype quickstart spark spark-streaming spark-sql demo oozie redis kafka

cdapio / cdap

spark-streaming,An open source framework for building data analytic applications.

Organization: cdapio

Home Page: https://cdap.io

unified integration platform dataset mapreduce spark spark-streaming java java-8 cdap

spark-streaming,A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

User: chabane

docker spark-sql scala kafka hbase parquet avro nodejs angular graphql

chermenin / spark-states

spark-streaming,Custom state store providers for Apache Spark

User: chermenin

Home Page: http://code.chermenin.ru/spark-states/

spark spark-streaming spark-structured-streaming structured-streaming state-store state stateful apache apache-spark

databrickslabs / dbldatagen

spark-streaming,Generate relevant synthetic data quickly for your projects. The Databricks Labs synthetic data generator (aka `dbldatagen`) may be used to generate large simulated / synthetic data sets for test, POCs, and other uses in Databricks environments including in Delta Live Tables pipelines

Organization: databrickslabs

Home Page: https://databrickslabs.github.io/dbldatagen

datagen pyspark python data-generation faker spark spark-streaming delta-live-tables deltalake databricks

dimajix / spark-training

spark-streaming,Repository used for Spark Trainings

Organization: dimajix

Home Page: http://www.dimajix.de

spark pyspark spark-streaming spark-ml hadoop hive sqoop scala python spark-training

dogukannulu / streaming_data_processing

spark-streaming,Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO

User: dogukannulu

airflow docker elasticsearch hadoop hdfs kafka kibana minio pyspark python spark spark-streaming zookeeper

dotnet / spark

spark-streaming,.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Organization: dotnet

Home Page: https://dot.net/spark

spark csharp dotnet analytics bigdata spark-streaming spark-sql machine-learning fsharp dotnet-core

edp963 / wormhole

spark-streaming,Wormhole is a SPaaS (Stream Processing as a Service) Platform

Organization: edp963

Home Page: https://edp963.github.io/wormhole

wormhole stream-processing spark-streaming

garystafford / kafka-connect-msk-demo

spark-streaming,For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR

User: garystafford

kafka kafka-connect spark-streaming spark aws kubernetes

harbby / sylph

spark-streaming,Stream computing platform for bigdata

User: harbby

Home Page: https://harbby.github.io/project/sylph/index.html

java sylph flink spark-streaming big-data sql streamsql

hortonworks / streamline

spark-streaming,StreamLine - Streaming Analytics

Organization: hortonworks

streaming real-time storm spark-streaming kafka kafka-streams flink

huangyueranbbc / spark_als

spark-streaming,基于spark-ml,spark-mllib,spark-streaming的推荐算法实现

User: huangyueranbbc

spark spark-mllib als spark-streaming spark-streaming-als

huangyueranbbc / sparkdemo

spark-streaming, spark全示例代码(java、scala) Spark most full instance code DEMO (java、scala)

User: huangyueranbbc

spark hadoop bigdata operator sparkjava spark-sql spark-streaming sparkfun-products sparkline sparkp

jaceklaskowski / spark-streaming-notebook

spark-streaming,Notes about Spark Streaming in Apache Spark

User: jaceklaskowski

spark spark-streaming apache-spark notebook

jacksu / utils4s

spark-streaming,scala、spark使用过程中，各种测试用例以及相关资料整理

User: jacksu

Home Page: http://jacksu.github.io/utils4s

spark scala-demo scala-spark json4s breeze scala akka spark-streaming

jmcmt87 / spark_app_twitter

spark-streaming,A data engineering project (Twitter monitor app)

User: jmcmt87

Home Page: https://share.streamlit.io/jmcmt87/streamlit_share/main/streamlit_share.py

kafka mongodb pyspark s3 spark-streaming altair pandas python

jubins / spark-and-mllib-projects

spark-streaming,This repository contains Spark, MLlib, PySpark and Dataframes projects

User: jubins

spark mllib spark-ml sparksql spark-streaming spark-dataframes pyspark python aws-ec2

kaantas / kafka-twitter-spark-streaming

spark-streaming,Counting Tweets Per User in Real-Time

User: kaantas

python spark spark-streaming apache-kafka twitter-api tweepy twitter pyspark

klarrio / open-stream-processing-benchmark

spark-streaming,This repository contains the code base for the Open Stream Processing Benchmark.

Organization: klarrio

Home Page: https://github.com/Klarrio/open-stream-processing-benchmark/wiki

stream-processing flink spark kafka-streams kafka distributed-systems distributed-computing real-time real-time-processing structured-streaming

learningjournal / spark-streaming-in-python

spark-streaming,Apache Spark 3 - Structured Streaming Course Material

User: learningjournal

Home Page: https://www.learningjournal.guru

apache-spark big-data bigdata data-lake pyspark python spark-sql spark-streaming

learningjournal / spark-streaming-in-scala

spark-streaming,Apache Spark 3 - Structured Streaming Course Material

User: learningjournal

Home Page: https://www.learningjournal.guru

apache-spark big-data bigdata datalake scala spark spark-sql spark-streaming

luckyzxl2016 / movie_recommend

spark-streaming,基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统

User: luckyzxl2016

spark-mllib spark-streaming ssm-maven scrapy scala hadoop nginx hive mysql

lw-lin / coolplayspark

spark-streaming,酷玩 Spark: Spark 源代码解析、Spark 类库等

User: lw-lin

apache-spark spark spark-streaming sparkcore structured-streaming

lw-lin / streaming-readings

spark-streaming,Streaming System 相关的论文读物

User: lw-lin

stream-processing streaming flink spark-streaming storm heron dataflow drizzle millwheel s4

martandsingh / apachespark

spark-streaming,This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.

User: martandsingh

apachespark data-analysis data-engineering database databricks datalake deltalake etl-pipeline hadoop hive

microsoft / data-accelerator

spark-streaming,Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

Organization: microsoft

spark spark-streaming spark-sql sparksql streaming-data streaming servicefabric nodejs docker hdinsight

microsoft / mobius

spark-streaming,C# and F# language binding and extensions to Apache Spark

Organization: microsoft

spark apache-spark rdd dataframe dstream dataset streaming csharp mobius kafka-streaming

mkuthan / example-spark

spark-streaming,Spark, Spark Streaming and Spark SQL unit testing strategies

User: mkuthan

spark spark-streaming testing

mkuthan / example-spark-kafka

spark-streaming,Apache Spark and Apache Kafka integration example

User: mkuthan

Home Page: http://mkuthan.github.io/blog/2016/01/29/spark-kafka-integration2/

spark spark-streaming kafka

nashtech-labs / real-time-stream-processing-engine

spark-streaming,This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

Organization: nashtech-labs

spark-streaming kafka elasticsearch apache-spark spark

osin-vladimir / architect_big_data_solutions_with_spark

spark-streaming,code, labs and lectures for the course

User: osin-vladimir

databricks databricks-notebooks aws-machine-learning spark-streaming etl data-analysis spark machine-learning python data-science

p7h / spark-mllib-twitter-sentiment-analysis

spark-streaming,:star2: :sparkles: Analyze and visualize Twitter Sentiment on a world map using Spark MLlib

User: p7h

Home Page: https://hub.docker.com/u/p7hb/

spark spark-streaming spark-mllib scala twitter-sentiment-analysis visualization machine-learning naive-bayes-classification

paypal / gimel

spark-streaming,Big Data Processing Framework - Unified Data API or SQL on Any Storage

Organization: paypal

Home Page: http://gimel.io

spark spark-streaming big-data gimel paypal data-api kafka cassandra hbase aerospike

qubole / kinesis-sql

spark-streaming,Kinesis Connector for Structured Streaming

Organization: qubole

Home Page: http://www.qubole.com

kinesis real-time-processing spark spark-streaming spark-structured-streaming structured-streaming

risingwavelabs / risingwave

spark-streaming,SQL engine for event-driven workloads. Perform streaming analytics, or build event-driven applications, real-time ETL pipelines, and feature stores in minutes. Unified streaming and batch processing. PostgreSQL compatible.

Organization: risingwavelabs

Home Page: https://www.risingwave.com/slack

database stream-processing cloud-native sql distributed-database rust serverless postgresql real-time postgres

snowch / movie-recommender-demo

spark-streaming,This project walks through how you can create recommendations using Apache Spark machine learning. There are a number of jupyter notebooks that you can run on IBM Data Science Experience, and there a live demo of a movie recommendation web application you can interact with. The demo also uses IBM Message Hub (kafka) to push application events to topic where they are consumed by a spark streaming job running on IBM BigInsights (hadoop).

User: snowch

cloudant python-flask-application spark spark-streaming hadoop kafka messagehub dsx notebook machine-learning