kubernetes-retired / spartakus Goto Github PK

[EOL] Anonymous Usage Collector

License: Apache License 2.0

Makefile 7.39% Shell 3.84% Go 87.50% Dockerfile 1.27%

spartakus's Introduction

Spartakus

Project Spartakus aims at collecting usage information about Kubernetes clusters. This information will help the Kubernetes development team to understand what people are doing with it, and to make it a better product.

What is in a report?
How do I run it?
What will we do with this information?
User documentation
Development
Future

Note: Spartakus does not report any personal identifiable information (PII). Anything that might be identifying, including IP addresses, container images, and object names are anonymized. We take this very seriously. If you think something we are collecting might be considered PII, please do let us know by raising an issue here.

Running Spartakus is a voluntary effort, that is, it is not baked into Kubernetes in any way, shape, or form. In other words: it operates on an opt-in basis—if you don't want to run it, you don't have to. If you want to run your own server and collect data yourself, you can do that, too—see also the user docs for more info on how to customize the reports.

What is in a report?

Reports include a user-provided cluster identifier, the version strings of your Kubernetes master, and some information about each node in the cluster, including the operating system version, kubelet version, container runtime version, as well as CPU and memory capacity.

An example report payload looks as follows:

{
    "version": "v1.0.0",
    "timestamp": "867530909031976",
    "clusterID": "2f9c93d3-156c-47aa-8802-578ffca9b50e",
    "masterVersion": "v1.3.5",
    "nodes": [
        {
            "id": "c8863d09ecc5be8d9791f72acd275fc2",
            "operatingSystem": "linux",
            "osImage": "Debian GNU/Linux 7 (wheezy)",
            "kernelVersion": "3.16.0-4-amd64",
            "architecture": "amd64",
            "containerRuntimeVersion": "docker://1.11.2",
            "kubeletVersion": "v1.3.2",
            "cloudProvider": "aws",
            "capacity": [
                {
                    "resource": "alpha.kubernetes.io/nvidia-gpu",
                    "value": "0"
                },
                {
                    "resource": "cpu",
                    "value": "4"
                },
                {
                    "resource": "memory",
                    "value": "15437428Ki"
                },
                {
                    "resource": "pods",
                    "value": "110"
                }
            ]
        },
        {
            "id": "5b919a15947b0680277acddf68d4b7aa",
            "operatingSystem": "linux",
            "osImage": "Debian GNU/Linux 7 (wheezy)",
            "kernelVersion": "3.16.0-4-amd64",
            "architecture": "amd64",
            "containerRuntimeVersion": "docker://1.11.2",
            "kubeletVersion": "v1.3.2",
            "cloudProvider": "aws",
            "capacity": [
                {
                    "resource": "alpha.kubernetes.io/nvidia-gpu",
                    "value": "0"
                },
                {
                    "resource": "cpu",
                    "value": "4"
                },
                {
                    "resource": "memory",
                    "value": "15437428Ki"
                },
                {
                    "resource": "pods",
                    "value": "110"
                }
            ]
        }
    ]
}

How do I run it?

To start using Spartakus in your cluster, use the following:

$ kubectl run spartakus \
    --image=gcr.io/google_containers/spartakus-amd64:v1.0.0 \
    -- volunteer --cluster-id=$(uuidgen)

This will generate a deployment called spartakus in your default namespace which sends a report once every 24 hours. It will report a random UUID as the cluster ID. Note that if you stop this deployment and re-run it, the UUID will be different. Managing the UUIDs is outside of the scope of Spartakus.

If you want to save the YAML manifest that the command above produces, you can simply execute the following (note: the --export flag strips cluster-specific information):

$ kubectl get deployment spartakus --export -o yaml

You needn't worry about CPU and memory usage of Spartakus, its resource usage footprint is minimal. If you're still concerned, you can edit the deployment to request a small share of CPU and memory; for example, Spartakus will work fine with 1m CPU and 10Mi mem on a five-nodes cluster.

What will we do with this information?

The information reported by Spartakus, in aggregate, will help the Kubernetes development teams prioritize our efforts. The better we understand what people are doing, the better we can focus on the important issues.

Development

Note that you only will need this section if you're contributing to Spartakus.

To build artefacts, we're using make. Simply run make or make test to build the binary in bin/; the container image build is carried out through Docker and hence requires you've got Docker running on your machine.

Future

Anything we add will follow the same strict privacy rules as outlined above. Some examples of things we would consider adding:

How many namespaces exist.
A histogram of how many pods, services, deployments, etc. exist on a per-namespace basis.
Average lifetime of namespaces, pods, services, etc.

spartakus's People

Contributors

Stargazers

Watchers

spartakus's Issues

Collect stats on namespaces

Number of namespaces
Histogram or average age of namespaces
Histogram or average number of (pods, services, deployments, jobs) per namespace

README: add incubator, community, and CoC sections

Please add in something like the last three section from here: https://github.com/kubernetes/kubernetes-template-project#community-discussion-contribution-and-support

Documenting the usage of database backend options

Currently there exists no documentation for database backend options, esp. around the HTTP option. Create user-friendly docs incl. example in a docs/ dir.

Original issue for clarity reasons:

Currently, the aggregation and reporting back-end is hardwired to BigQuery. It would be very beneficial, for example for on-prem deployments or individuals who don't want to or can not use BQ to make Spartakus support different back-ends in a pluggable manner. For example, one might want to use Minio+PrestoDB or simply a RDBMS such as Postgres.

Collect resource utilization metrics

Least detailed: per-node sum of pods' requests and limits (capacity is already provided)
More detailed: for each node, a list of the pods' requests and limits
Even more detailed: also per-node and per-pod usage (collected from heapster)

Also, Future says "A histogram of how many Pods, Services, Deployments, etc. exist per-Namespace" and I'm not sure if this is implied, but I think knowing the replica count for Deployments and ReplicaSets would be useful.

Expired/invalid TLS certificate

Upon running spartakus, the logs show:

E1229 17:26:30.118403       1 volunteer.go:68] failed sending report: HTTP request failed: Post https://spartakus.k8s.io/api/v1: x509: certificate has expired or is not yet valid

List of things I'd like to know about how people are using Kubernetes

I'm very excited to see this project. I've been mentioning around the office for weeks that the community needs a better idea of how K8s is actually being used.

I'd like to help expand the project to collect more useful information about installations. Here's a dump of the things I've been wondering about that we could collect. I'd be willing to add any of these or help in the collection & anonymization of other datapoints.

Context (these would likely need to be provided in plain text at launch time, so wouldn't want to ask a lot)

Phase of cluster (e.g. prototype, development, staging, production)
Installed in private datacenter or (specific) public cloud?

Cluster setup

Master node count
Etcd node count
Service node count
Node volatility, how quickly are nodes added or removed from the cluster?
Node consistency [specifically I wonder if master/etcd nodes are sized the same as other nodes]
Are Etcd nodes also worker nodes (cf self hosting)

Networking

Block size of Pod & Service networks
Is an overlay network being used
Is policy enabled
Is CNI being used
What, if any, SDN is in use

Disk settings

How big/fast are the disks used to host nodes?
What are the log rates of the kubernetes components?

Consider supporting flags outside of deployment

So that we can easily upgrade the deployment, maybe we could put the flags outside of the container?

I am thinking of a componentconfig-style typed object, that lives either in a ConfigMap or simply a well known annotation on the kube-system namespace.

In addition, we could then provide additional metadata... for example the installation tool could include a set of tags ("aws kops")

Add `timestamp` to all reports

Currently, the timestamp field is only added to reports that are processed with a Spartakus collector. If you run the volunteer with --database=stdout or even --database=bigquery://... your records will have no timestamp.

Create a SECURITY_CONTACTS file.

As per the email sent to kubernetes-dev[1], please create a SECURITY_CONTACTS
file.

The template for the file can be found in the kubernetes-template repository[2].
A description for the file is in the steering-committee docs[3], you might need
to search that page for "Security Contacts".

Please feel free to ping me on the PR when you make it, otherwise I will see when
you close this issue. :)

Thanks so much, let me know if you have any questions.

(This issue was generated from a tool, apologies for any weirdness.)

[1] https://groups.google.com/forum/#!topic/kubernetes-dev/codeiIoQ6QE
[2] https://github.com/kubernetes/kubernetes-template-project/blob/master/SECURITY_CONTACTS
[3] https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance-template-short.md

Add Cloud Provider to Reports

The cloud provider name is not PII and will provide useful insight into how people are using Kubernetes. This information can be found on a per-node basic from node.Spec. Consuming this change will require updating the database schema.

[Feature Request] Make imagepull always and have it reboot nightly

Will enable updating

self-updating deployments

It would be great if spartakus would be able to update its own deployment and ideally we could roll out the update in a controlled manner with a feedback loop. Also, it would be cool if people could subscribe to alpha/beta/stable reporting channels.

We are building something that plugs into an Omaha server for the etcd and Prometheus Operator. Would that be OK for spartakus?

Tag v1.1.0 and build new images

@thockin I'd like to release v1.1.0 of Spartakus given the new features that were added recently, namely:

kubernetes client bump to 1.5.1;
cloud provider reporting; and
support for extensions reporting.

Thoughts? Is there anything else you'd like to accomplish before tagging a new version?

BigQuery data available?

Is the BigQuery data available somewhere for others to search through?

I didn't see a link anywhere but would like to query the data for some research.

Move to Go dep

Currently, Spartakus is using Godeps, it would be rad to move to dep.
I'm happy to take care of this, if desired.

Inquiry: Intention of Timestamp structure

In pkg/volunteer/volunteer.go the Timestamp collected is being cast to a string

https://github.com/kubernetes-incubator/spartakus/blob/633430c4022c62dc1046682c7f3d3976eb783c60/pkg/volunteer/volunteer.go#L123

This is similarly defined in the schema pkg/database/bigquery.schema.json:

https://github.com/kubernetes-incubator/spartakus/blob/633430c4022c62dc1046682c7f3d3976eb783c60/pkg/database/bigquery.schema.json#L7-L11

As the service is primarily defined to utilize BigQuery as a datastore it seems odd to utilize a string versus a TIMESTAMP (or even an INTEGER since BigQuery stores this as int64).

Was the use of String an intentional design decision? What are the ramifications of changing this to a field of type Timestamp?

report metrics from third-party resource

It would be nice for people to report additional data without changing spartakus source code. I propose that we create a new third-party resource type called "SpartakusData" that can be used to report additional arbitrary data.

It would look something like this:

apiVersion: "k8s.io/v1"
kind: "SpartakusData"
metadata:
  name: "kube-dns"
spec:
  queries: 10040
  version: "v20"

This would be reported out in the payload like:

{
    "clusterID": "2f9c93d3-156c-47aa-8802-578ffca9b50e",
...
    "data": {
      "kube-dns": {"queries": 10040, "version": "v20"}
     }

Example use cases:

Reporting Cloud Provider specific details like number of persistent disks allocated
Allow add-ons to report usage data e.g. how many DNS queries from kube-dns
Track install specific metrics like other component versions

cc @squat @sym3tri

Archive this project

It does not appear like we are doing much with this data. Let's turn it down in the utility cluster and archive this project.

/assign @fejta @thockin