lightbend / fdp-sample-applications Goto Github PK
View Code? Open in Web Editor NEWAll sample applications for Fast Data Platform
License: Apache License 2.0
All sample applications for Fast Data Platform
License: Apache License 2.0
Hello,
I'm currently working with flink example in project.
But after activate kafka/zookeeper server and start ingest
:
sbt:ingestRun> ingest
[info] Running (fork) com.lightbend.fdp.sample.flink.ingestion.DataIngestion
[info] Got /Users/jung/workspace/etc/fdp-sample-applications/apps/flink/source/core/data/nycTaxiFares.csv to process
[info] Got /Users/jung/workspace/etc/fdp-sample-applications/apps/flink/source/core/data/nycTaxiRides.csv to process
...it does not go further after this line.
I'm running in local system(not using docker, DC/OS...), and my config is:
dcos {
kafka {
brokers = "localhost:9092"
brokers = ${?KAFKA_BROKERS}
group = "group"
group = ${?KAFKA_GROUP}
intopic = "taxiin"
intopic = ${?KAFKA_IN_TOPIC}
outtopic = "taxiout"
outtopic = ${?KAFKA_OUT_TOPIC}
zookeeper = "localhost:2181"
zookeeper = ${?ZOOKEEPER_URL}
## settings for data ingestion
loader {
sourcetopic = ${dcos.kafka.intopic}
sourcetopic = ${?KAFKA_IN_TOPIC}
directorytowatch = "/path/to/fdp-sample-applications/apps/flink/source/core/data"
directorytowatch = ${?DIRECTORY_TO_WATCH}
pollinterval = 1 second
}
}
}
I fetched taxi data from http://training.data-artisans.com/exercises/taxiData.html , and put files in ./apps/flink/source/core/data
.
Is there something missing?
Thanks.
I'm currently evaluating FDP. In the process, I have attempted to install the killrweather app as described here: https://developer.lightbend.com/docs/fast-data-platform/1.3.1/user-guide/sample-apps/index.html#fast-data-platform-killrweather
The documentation points to the README.MD
in this repository: https://github.com/lightbend/fdp-sample-applications/blob/develop/apps/killrweather/README.md
That file instructs me to deploy the killrweather app using this command:
dcos marathon app add killrweather-app/src/main/resources/killrweatherAppDocker.json
I'm assuming this refers to https://github.com/lightbend/fdp-sample-applications/blob/develop/apps/killrweather/source/core/killrweather-app/src/main/resources/killrweatherAppDocker.json. I think that's right, because installing the loader worked with a small modification (see below).
However, there are a couple of issues with this JSON file:
process-templates.sh 1.3.1
, as instructed, does not update the file.CASSANDRA_HOSTS
: a comma is missing at the end of the line. The same problem exists in killrweatherloaderDocker.json
and the corresponding .json.template
files.Error while fetching [http://fdp-dcos-elasticl-5us6x2035mlk-426186471.eu-west-1.elb.amazonaws.com/service/marathon/v2/pods]: HTTP 422: "{"message":"Invalid JSON","details":[{"path":"/containers","errors":["'containers' is undefined on object: <snipped by me>
Indeed, the killrweatherloaderDocker.json
specifies a containers
array with two extensive elements, whereas killrweatherAppDocker.json
only has a rather simplistic container
object.
I am unfamiliar with DC/OS, so I am unsure how to fix this.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.