It is shared in Google Presentations (see the link on Slack)

Hi,
I would like to know whether we need the vendor/ folder in the repository.
I will suggest we remove it from the repo, add it to .gitignore since its automatically generated after a dep ensure

We recently migrated our monitoring for edgecast and chinacache to microservices exposing data in prometheus format. Read the article on our techblog.
In order to make our infrastructure more homogenous, it would be nice to have the same metric format everywhere. Therefore it would be great, if this service could also expose metrics in prometheus format.

• read API documentation
• provide links to the relevant documentation

Test if the data is sent properly

• Learn about unit tests and integration tests
• Learn about test driven development
• Get a unit test for Akamai (a .txt file)

Terms we need to define:

• daemon
  Process that runs in the background and provides a service that is not visible to the user (mail server, ssh server, monitoring).
• sample data
  Data provided for testing purposes
• Kafka
  Distributed Message Queue used at trivago
  Can store any type of message.

What is the task?
Prepare simplified daemon, which takes file and sends to somewhere (e.g. Kafka)

What data do we need?
Test File for Akamai
Real world data

Who provides the data?
Test file -> trivago
Real world data -> credentials from Akamai

Who will implement the daemon
Pair programming first
Maybe separation later

When to implement
Today: Golang tutorial for ramp-up
Tomorrow: Test-driven development of daemon using sample data
Later: Integrate with live data

Where to store data?
First: standard-out (commandline output)
Later: Kafka

Where to host our solution?
First: on our laptops using installed golang
Later: trivago's infrastructure
Docker?
OpenNebula VM
Bare metal

Why store in Kafka?
Kafka provides buffering
Already set-up
Used for other parts of trivago's infrastructure
Integrated with InfluxDB and Grafana

Why not write our own dashboard?
Grafana already provided
Separation of concerns

Why not to use go?
No previous knowledge

Why do we use go?
Fast (Based on C)
"Easy"
Good language to run on server
Low memory usage
Concurrency
Low overhead -> no Runtime
Monitoring will not be interrupt because of garbage collector Fast garbage collector
Type safe language -> errors visible at compile time and not at runtime (safes debugging time)
Static type system -> defining inputs and outputs of a function locally
very good external tooling (debugger, unit testing, gofmt, govet...)
big ecosystem (Kubernetes, Docker, CoreOS, InfluxDB all written in Go)
part of trivago's ecosystem

Documentation
Provide link to docs
Akamai API Overview
read functions
read documentation
understand which functionality it provides
Golang docs
Go tutorial
Hello world go app on local machine
Hello world go server

Providing test harness
Introduction to unit tests
Difference between unit and integration tests
Test-driven development
Unit tests File for Akamai

Prototyping
Understanding what we have to send to Kafka
testing if it works

Real world scenario
Change config files so that we send actual data to our server
Configure Docker
What data do we send to Kafka "in the real world"
test if we can use our go script to send the data to kafka
Create dashboard for our output data

Adjust the Akamai CDN settings to use our monitoring endpoint.

See test data provided in #5.

We need concrete knowledge to configure and work with a docker on the server which will serves our go script afterward.

List of tasks should be done here

• Golang docs
• Go tutorial
• Hello world go app on local machine
• Hello world go server

We have a number of sample requests from the Akamai Cloudwatch API.
There are unknown fields in the json request payload.
We need to define the purpose of the properties before we can use them for our monitoring daemon.

For #7 we need to host our service on a publicly available address.
We have the following options for deployment:

Running the daemon on trivago's infrastructure

• Docker? Is it a good solution?
• Or? OpenNebula VM
• Or? Bare metal

For our project we need a working Docker setup.

Requirements:

1. Learn the basics of Docker. Here and here are two good beginner tutorials
2. Install Docker
3. Run a "hello world" container (Command: docker run --rm hello-world)
4. Create a container, which runs Golang inside. Here is a tutorial.
5. Teach the other team members the basics of Docker.
6. Add a Dockerfile to our project and push it to Github.

• Understanding the input in Kafka
• Sending the output of the daemon directly to Kafka

Write a golang program that uses the provided fixtures of the Akamai CDN response for unit testing.

• Running the daemon prototype on our laptops using installed golang