Giter VIP home page Giter VIP logo

sdk's Introduction

Grafana SDK Go Report Card

SDK for Go language offers a library for interacting with Grafana server from Go applications. It realizes many of HTTP REST API calls for administration, client, organizations. Beside of them it allows creating of Grafana objects (dashboards, panels, datasources) locally and manipulating them for constructing dashboards programmatically. It would be helpful for massive operations on a large set of Grafana objects.

It was made foremost for autograf project but later separated from it and moved to this new repository because the library is useful per se.

Library design principles

  1. SDK offers client functionality so it covers Grafana REST API with its requests and responses as close as possible.
  2. SDK maps Grafana objects (dashboard, row, panel, datasource) to similar Go structures but not follows exactly all Grafana abstractions.
  3. It doesn't use any logger, instead API functions could return errors where it need.
  4. Prefere no external deps except Go stdlib.
  5. Cover SDK calls with unit tests.

Examples GoDoc

	board := sdk.NewBoard("Sample dashboard title")
	board.ID = 1
	board.Time.From = "now-30m"
	board.Time.To = "now"
	row1 := board.AddRow("Sample row title")
	row1.Add(sdk.NewGraph("Sample graph"))
	graph := sdk.NewGraph("Sample graph 2")
	target := sdk.Target{
		RefID:      "A",
		Datasource: "Sample Source 1",
		Expr:       "sample request 1"}
	graph.AddTarget(&target)
	row1.Add(graph)
	grafanaURL := "http://grafana.host"
	c := sdk.NewClient(grafanaURL, "grafana-api-key", sdk.DefaultHTTPClient)
	response, err := c.SetDashboard(context.TODO() ,*board, sdk.SetDashboardParams{
		Overwrite: false,
	})
	if err != nil {
		fmt.Printf("error on uploading dashboard %s", board.Title)
	} else {
		fmt.Printf("dashboard URL: %v", grafanaURL+*response.URL)
	}

The library includes several demo apps for showing API usage:

You need Grafana API key with admin rights for using these utilities.

Installation Build Status

Of course Go development environment should be set up first. Then:

go get github.com/grafana-tools/sdk

Dependency packages have included into distro. govendor utility used for vendoring. The single dependency now is:

go get github.com/gosimple/slug

The "slugify" for URLs is a simple task but this package used in Grafana server so it used in the SDK for the compatibility reasons.

Status of REST API realization Coverage Status

Work on full API implementation still in progress. Currently implemented only create/update/delete operations for dashboards and datasources. State of support for misc API parts noted below.

API Status
Authorization API tokens and Basic Auth
Annotations partially
Dashboards partially
Datasources +
Alert notification channels +
Organization (current) partially
Organizations partially
Users partially
User (actual) partially
Snapshots partially
Frontend settings -
Admin partially

There is no exact roadmap. The integration tests are being run against the following Grafana versions:

With the following Go versions:

  • 1.14.x
  • 1.13.x
  • 1.12.x
  • 1.11.x

I still have interest to this library development but not always have time for it. So I gladly accept new contributions. Drop an issue or contact me.

Licence

Distributed under Apache v2.0. All rights belong to the SDK authors. There is no authors list yet, you can see the full list of the contributors in the git history. Official repository is https://github.com/grafana-tools/sdk

Collection of Grafana tools in Golang

sdk's People

Contributors

aborilov avatar aleksanderaleksic avatar azun avatar bobheadxi avatar bradbeam avatar colega avatar giedriuss avatar github-vincent-miszczak avatar grafov avatar hjet avatar indragunawan avatar jd4900 avatar jorgelbg avatar jpdstan avatar k-phoen avatar kevitan avatar krya-kryak avatar masudur-rahman avatar mcristina422 avatar ntindall avatar phillebaba avatar pkbhowmick avatar ptxmac avatar raffaelschmid avatar safaci2000 avatar tdabasinskas avatar tonglil avatar y3llowcake avatar yasumoto avatar zibuyu28 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sdk's Issues

Missing default values for Time breaks grafana 6.7.2

The Board type contains a Time structure, which has two strings in it. If you don't fill those strings in, and you try to just create a simple board (following roughly the sample code in the README), the call succeeds but the resulting board gives a javascript error because Grafana thinks the board has a timerange but there are no from or to values.

Specifically, on line 55 of grafana's template_srv.ts, we have:

     const from = this.timeRange.from.valueOf().toString();

The structure this.Timerange is not null, but this.timeRange.from is undefined.

This means that very simple code like:


	board := grafana.NewBoard("MY TITLE")
	row := board.AddRow("MY ROW")
	row.Add(grafana.NewGraph("MY GRAPH"))
	response, err := client.SetDashboard(context.Background(), *board, grafana.SetDashboardParams{
		Overwrite: true,
	})

creates a board that breaks Grafana's front end.

This is more a bug with Grafana's API than anything else, but the SDK should work around it IMO.

🐛 : Group vs GroupBy

My recent pull request (#180) modified the Target.GroupBy field to be singular, but as it happens, what I needed was to change the Target.Group to Target.GroupBy. At least in Grafana 8, the schema for Target.GroupBy should be what the SDK currently defines as the schema for Target.Group - is there any major issue with merging the two, or at least having identical schema declarations for them?

cannot unmarshal object into Go struct field Board.panels of type string

func (m *manager) DashboardPanels(ctx context.Context, uid string) ([]*grafanasdk.Panel, error) {
	board, _, err := m.grafanaClient.GetDashboardByUID(ctx, uid)
	if err != nil {
		return nil, err
	}
	return board.Panels, nil
}

So strange, i have checked my dashboard json file, the panels field, it is an array not a string

Missing panel types for timeseries and state-timeline

When running cortextool analyse grafana (as desribed here) the panels on my dashboard came back empty in the metrics-in-grafana.json output.

I traced it down to this library which is vendored into grafana/cortex-tools. All my panels are of type "timeseries" and "state-timeline", which aren't implemented in sdk/panel.go.

Disappearing constant variables after getting and setting the same dashboard

I use grafana v8.0.2

My dashboard has following json model:

... "templating": { "list": [ { "datasource": null, "description": null, "error": null, "hide": 2, "includeAll": false, "label": "", "multi": false, "name": "period", "query": "1d", "refresh": false, "regex": "", "skipUrlSync": false, "type": "constant" } ] }, ...

After that code my dashboard has constant variables lost

board, properties, err := client.GetDashboardByUID(context.Background(), grafanaDashboardUid)
if err != nil {
	log.Fatal(err)
}

_, err = client.SetDashboard(context.Background(), board, sdk.SetDashboardParams{
	FolderID: properties.FolderID,
	Overwrite:  true,
})
if err != nil {
	log.Fatal(err)
}

Json model:
"templating": { "list": [] },

But if i use raw functions everything is fine

boardJson, properties, err := client.GetRawDashboardByUID(context.Background(), grafanaDashboardUid)
if err != nil {
	log.Fatal(err)
}

_, err = client.SetRawDashboardWithParam(context.Background(), sdk.RawBoardRequest{
	Dashboard:  boardJson,
	Parameters: sdk.SetDashboardParams{
		FolderID:   properties.FolderID,
		Overwrite:  true,
	},
})
if err != nil {
	log.Fatal(err)
}

I've found out that Options field of TemplateVar struct should have omitempty tag not to cause disappearing constant variables.
https://github.com/grafana-tools/sdk/blob/master/board.go#L88

Without omitempty tag Grafana sees that my constant variable has options json field and doesn't apply dashboard correctly.

But why Grafana has such behavior i don't know.

Start using generics & Go 1.18

It has been an issue for a long time to use this SDK with structured data in Grafana. There are two parts to it:

  • There is no way to express certain things in Go easily that we can do with JSONs. Go 1.18 is around the corner which has support for generics so I propose moving to that ASAP
  • Lack of schema in Grafana. Even though you can generate any kind of value with the SDK but does that value make sense to Grafana itself? This ideally should be solved in Grafana with Cue or something? Also, perhaps we could automatically generate what values don't make sense by generating random dashboards and checking whether there are any JS errors in the console when opening that dashboard?

Roadmap/Additional Endpoint Implementation

Hi,

I'm looking into the possibility of switching the Grafana provider for Terraform to use this SDK instead of an independent client, however it doesn't appear that this SDK includes any implementation for Grafana Alert Notifications. Is this on the roadmap to be implemented at all in the near future?

We'd be interested in contributing to the SDK as well to further it's development! However, is there anyone actively maintaining it/reviewing PRs at this time?

Thanks,
Malcolm

OrgUser expecting `id` instead of `userId`

sdk/rest-org.go

Line 227 in c3f3511

func (r *Client) GetOrgUsers(ctx context.Context, oid uint) ([]OrgUser, error) {

 {
        "avatarUrl": "/avatar/46d229b033af06a191ff2267bca9ae56",
        "email": "admin@localhost",
        "lastSeenAt": "2021-03-11T12:17:39Z",
        "lastSeenAtAge": "< 1m",
        "login": "admin",
        "name": "",
        "orgId": 3,
        "role": "Admin",
        "userId": 1
    },

this API returns userId field, but we expecting id here

sdk/org.go

Line 28 in c3f3511

ID uint `json:"id"`

so ID will always has 0 value

Support /import endpoint

This was triggered from a conversation I had with grafana folks:

If you want to share dashboards among different grafana instances then the endpoint to use is the /import one. This endpoint tries to create all the library panels that do not already exist in the new instance. If a library panel with the same UID already exists in the new instance then it will ignore it. This was implemented as part of grafana/grafana#39214 work. For more details please check the implementation there.

It seems like there is a use case where we should have the /import endpoint in the SDK.

Target has no measurement field

There is no Measurement field in the Target structure which forces a measurement to be dropped when retrieving a dashboard JSON.

When unmarshalling a JSON created in the UI that uses InfluxDB as it's datasource the measurement field is dropped. The current struct lists various fields for various types of datasources but does not have a section for fields for InfluxDB.

Are there any other fields that should be added? I've added the measurement field in my own fork here but am wondering if there are any fields I am missing.

Everytime i use c.SetDashboard(context.TODO(), *board, sdk.SetDashboardParams, i am getting Templating init failed Cannot read property 'valueOf' of undefined

func main () {
	
        board := sdk.NewBoard("Sample dashboard title1234")
	board.ID = 1
	row1 := board.AddRow("Sample row title")
	row1.Add(sdk.NewGraph("Sample graph"))
	graph := sdk.NewGraph("Sample graph 2")
	target := sdk.Target{
		RefID:       "A",
		Datasource:  "Sample Source 1",
		Hide:        false,
		Measurement: "sample request 1"}
	graph.SetTarget(&target)
	row1.Add(graph)
	grafanaURL := "http://localhost:3000"
	c := sdk.NewClient(grafanaURL, "admin:xxxxx", sdk.DefaultHTTPClient)
	response, err := c.SetDashboard(context.TODO(), *board, sdk.SetDashboardParams{
		Overwrite: false,
	})
	if err != nil {
		fmt.Printf("error on uploading dashboard %s", board.Title)
	} else {
		fmt.Printf("dashboard URL: %v", grafanaURL+*response.URL)
	}
}```


dashboard is getting created but its giving the below error.  Need help on please

"Templating
Templating init failed Cannot read property 'valueOf' of undefined

An unexpected error happened
Details"

Example JSON doesn't work with Grafana 4.1

Summary: The Example produces JSON that can not be imported by Grafana 4.1

I'm trying to write a program that posts a dashboard to my new Grafana 4.1 instance, but I can't get it working. The JSON returned from the server is "[]". I reproduced this issue using the Example JSON given in the docs, and through trial and error using the interactive JSON import narrowed the problem down to this part:

      "grid": {
            "leftLogBase": null,
            "leftMax": null,
            "leftMin": null,
            "rightLogBase": null,
            "rightMax": null,
            "rightMin": null,
            "threshold1": null,
            "threshold1Color": "",
            "threshold2": null,
            "threshold2Color": "",
            "thresholdLine": false
          },

Including the "grid" key at all (even without sub-keys) causes the dashboard to fail after import, and I assume it's the same issue when doing it programmatically. I don't know enough about Grafana to say why this is the case or what the right resolution is. However, it does seem weird because looking at the JSON for existing dashboards shows the same "grid" key. So maybe this is a red herring.

StatusMessage Datasource Bug

Hello,

So the response being returned from: CreateDatasource is StatusMessage unfortunately that does not map out correctly.

The response for the current version of the API looks like this for: Version 8.3.6

{
  "datasource": {
    "id": 5,
    "uid": "EwLFuaBnk",
    "orgId": 1,
    "name": "testing",
    "type": "elasticsearch",
    "typeLogoUrl": "",
    "access": "direct",
    "url": "http://1.2.3.4",
    "password": "",
    "user": "",
    "database": "grafanasdk",
    "basicAuth": false,
    "basicAuthUser": "",
    "basicAuthPassword": "",
    "withCredentials": false,
    "isDefault": false,
    "jsonData": {
      "esVersion": "5",
      "maxConcurrentShardRequests": "256",
      "timeField": "@timestamp"
    },
    "secureJsonFields": {},
    "version": 1,
    "readOnly": false
  },
  "id": 5,
  "message": "Datasource added",
  "name": "testing"
}

Except for id, message and name none of the other fields will be mapped correctly. As StatusMessage is being used by a variety of fields, before I attempt to fix it, I was wondering what approach would you like to take?

I saw a Generics 1.18 ticket so not sure if this would benefit from it.

I was thinking of a few approaches.

  1. Parse the the message twice giving it raw JSON and a 2nd time with just the content of datasource ?
  2. Create a new DatasourceStatusMessage.
  3. Look at how Generics might come into play to create a more generic approach.

Any suggestions on how you'd prefer to have this handled?

feature request: Could we have "Data source proxy calls" in this sdk?

I found an API that is not added to this SDK yet.

Could we have a method like
func (r *Client)ProxyDatasource(ctx context.Context, datasourceID, rawRequest string) (rawReply string, err error)
to support this API in this SDK?

Could I create a PR to add this feature if it is approved?

Thanks very much.

Add "Tags" field to Target

Currently, the Target type has no fields for Tags, which is something that we use in our queries for separating out aggregated data in single panels. tags does appear as a field in the Panel JSON, and it would be very useful for us to have that.

Cannot create new datasource: cannot unmarshal array into Go value of type sdk.StatusMessage

Hi
I'm trying to create a new datasource using the SDK, here the code:

status, err = grafanaClient.CreateDatasource(ctx, requestedDatasource)

The type of the requestedDatasource is sdk.Datasource and the error is:

json: cannot unmarshal array into Go value of type sdk.StatusMessage

Here is a sample content for requestedDatasource:

{0 0 yashar prometheus  http://172.16.78.68:9090 <nil> <nil> <nil> <nil> <nil> <nil> false <nil> <nil>}

c.GetActualOrg undefined (type *sdk.Client has no field or method GetActualOrg)

Hi, there may some issue needs be help

The code below

	c := sdk.NewClient(BaseURL, APIKey, g.httpClient)
	if _, err := c.GetActualOrg(); err != nil {
		err = errors.Wrapf(err, "connection to grafana failed")
		logrus.Error(err)
		return err
	}

Got error:

c.GetActualOrg undefined (type *sdk.Client has no field or method GetActualOrg)

And I saw the README.md with go version support

1.13.x
1.12.x
1.11.x

My go version

go version go1.14.1 darwin/amd64

Is there any plan to support the latest go version? Maybe I can contribute to support the latest go version

Status of Project?

I see several PRs and tickets that are months old and I also had a few commits in that have been pending for a while. I understand that everyone has their lives and such that keeps them busy. I'm just curious if maybe you guys are looking for help? or what the interest is in the sdk at this point?

I ended up forking the project and using the fork in my own project since the PRs were taking too long to get in but this seems like a pretty active project so I'd love to be able to contribute back to it and help if I can. Particularly since work has an interest in the tools i'm developing with SDK so I can do spend some paid cycles on this.

Any updates would be appreciated.

Generate dashboard failed

Grafana Version: v7.0.3
Test code:

board := sdk.NewBoard("Sample dashboard title")
c := sdk.NewClient("<https url>", "<api key with admin role>", sdk.DefaultHTTPClient)
	response, err := c.SetDashboard(context.TODO() ,*board, sdk.SetDashboardParams{
		Overwrite: false,
	})
if err != nil {
	fmt.Printf("error on uploading dashboard %s", board.Title)
} else {
	fmt.Printf("dashboard URL: %v", grafanaURL+*response.URL)
}

Error:
image

Required Version API Call

This is more of a conversational starter than an issue, but I was wondering if there was a way to implement a behavior where we can tag the various API calls with a minimum/max version.

Aka we know that. client.GetHealth() requires grafana version X, so if grafana version is set either through a init method, or via an API call where state is populated, we would fail the SDK call more gracefully.

./bin/foobar server info
Error: /api/health requires Grafana version 1.2.3, you are connected to a server running version 0.1.2 which is lower than the minimum required version

Any thoughts?

add support for gauge panel type

Currently, if we try to create a dashboard using json model containing a gauge panel, it will be marshalled as a custom panel instead of a gauge panel. Due to json nesting of custom panel, targets.expr for the gauge panel will not be parse and no data is displayed on the gauge.

Proposal: Add GaugePanel struct in panel.go

panel UnmarshalJSON missing row

Hi,When I import the dashboard, the collapsed panel is missing
image

The reason is rowPanel json.Unmarshal customPanel

After adding the above code, it can be imported normally. Why is rowPanel missing?
image

Proposal: Please start using Semantic Versioning

I found that this project already supports Go modules. But sadly, the [tags](https://github.com https://github.com/grafana-tools/sdk/tags) doesn't follow Semantic Versioning, which means that all tags of this project will be ignored by Go modules and replaced by pseudo-versions, go get acts weirdly when tags are not in that form. It would be great to have the tagged release be named in the format vX.X.X format so that go mod can read it.

	github.com/grafana-tools/sdk v0.0.0-20200802204627-e6a8cdd8fef5

Else the mod file shows something like github.com/grafana-tools/sdk v0.0.0-20200802204627-e6a8cdd8fef5 which is not very readable and difficult to upgrade. It’s hard to verify which version is in use. This is not conducive to version control.

So, I propose this project to follow Semantic Versioning in future versions. For example, v1.0.1, v2.0.0, v3.1.0-alpha, v3.1.0-beta.2etc, so that other project can use tag in go.mod.

Target: Alias vs AliasBy; Group vs GroupBy

These 2 groups of fields seem remarkably similar, and I'm not sure which to use. are both needed? It seems that at least for Target.Group vs Target.GroupBy, the former has a little more type safety, but is a little clunkier to use. It seems also that if I declare a Tag to group on, it does not end up in the query, while it does show up in the panel JSON:

{
  "params": [
    "k8s_cluster"
  ],
  "type": "tag"
},

Query:
... GROUP BY time(1s) fill(null)

For Target.Alias vs Target.AliasBy, I don't know if it's working properly, because the legend does not show the series breakdown in the legend. Again, the syntax of these two is different, but the effect seems to be the same in the panel JSON, though it doesn't seem to make a difference in the query.
"aliasBy": "$tag_k8s_cluster" in the panel JSON has no effect on the query for the panel.

When using the flux query in graph panel targets, grid are not displaying.

I am using Flux 2.X as the datasource.

I have defined the targets in my code like this..

board := sdk.NewBoard("sample")
board.ID = 1
row1 := board.AddRow("Sample row title")
// row1.Add(sdk.NewGraph("Sample graph"))
graph := sdk.NewGraph("Sample graph 2")
strPointer := sdk.MixedSource
graph.Datasource = &strPointer
target := sdk.Target{
	RefID:      "A",
	Datasource: "Influx 2.0",
	Query:      "Flux Query"}
//Query: "select * from"}
graph.AddTarget(&target)
graph.GraphPanel.Xaxis.Show = true
graph.GraphPanel.Yaxes = append(graph.GraphPanel.Yaxes, sdk.Axis{Show: true, Format: "short", LogBase: 1, Decimals: 0})
graph.GraphPanel.Yaxes = append(graph.GraphPanel.Yaxes, sdk.Axis{Show: true, Format: "short", LogBase: 1, Decimals: 0})
row1.Add(graph)

when the post call is made, i can able to see the dashboard and panel created, and targets added successfully with the query and datasource thats defined.. but i am not able to see the graph panel with grid and graph..

When i used the "Expr" instead of "Query" in the targets.. i can able to see grid coming, but the query is not visible..

is there a bug or sdk not not supporting Flux queries?

I am using grafana 7.3.3 and

Make a new release for `sdk`

There are a lot of commits since the last release.
Can you guys please make a new release for the following sdk so that various modules upgrading tools may keep the dependency up-to-date ?

edit pannel error: TypeError: Cannot read property 'length' of null

grafana version: 8.4.1
I create a dashboard and an empty timeseries pannel:

`board := sdk.NewBoard(d.DashName)
board.Time.From = "now-15m"
board.Time.To = "now"

panel := sdk.NewTimeseries("empty panel")
panel.Editable = true

board.Panels = append(board.Panels, panel)
dashRes, err := c.SetDashboard(ctx,*board, sdk.SetDashboardParams{
Overwrite: false,
})`

after i create it , I can see it in grafana, but when i edit the pannel, failed:
image

Dashboard search results should include what folder they belong to.

When searching for dashboards, the current API will include information about the dashboard if it belongs to one that is not the "default" folder. This is what the JSON looks like.

{
"id":1,
"uid":"<redacted>",
"title":"Jobs Dashboard",
"uri":"<redacted>",
"url":"/dashboards/d/<redacted>/jobs-dashboard",
"slug":"",
"type":"dash-db",
"tags":[],
"isStarred":false,
"folderId":34,
"folderUid":"<redacted>",
"folderTitle":"Test Folder",
"folderUrl":"/dashboards/dashboards/f/<redacted>/test-folder"
}

The FoundDashboard interface should probably include that information.

Setup consistent way for optional parameters (like `page` and `perpage`) through all the SDK functions

It is about other ways to solve the problem fixed in #46.

There are a lot of functions in Grafana API that could accept page and perpage as optional params. In a number of functions, there is also query parameter. I think it would be good for clarity to setup them all in a consistent way through all the SDK. It could be reached in a number of ways:

  1. Add page and pergage explicitly to the functions where it applicable. Additionally, define the functions with All suffix like GetAllUsers that would get whole object lists without paging parameters.
  2. Setup page and perpage as optional slice like that: GetUsers(paging ...int). It may be not obvious though and requires a careful explanation in the function comments.
  3. Pass optional functional parameters to all the functions of the SDK where optional params allowed. So it would allow pass them in any order:
sdk.GetUsers()
sdk.GetUsers(sdk.Page(200))
sdk.GetUsers(sdk.Query("match"), sdk.Page(1), sdk.Perpage(10))

It could be refactored for the next major release with breaking changes. So the current state could be fixed with v1.0 (it has 0.x numeration yet). And the change proposed above marked with v2.0

I don't decide that variant would be better for the API clarity yet. The latest way with funcparams looks more attractive for me.

Add ability to add Links to dashboards

Current dashboards have a field Links which is a slice of a private field link. Because this field is private, we're not able to construct new links to add to dashboards. Is there any reason why this struct isn't exported? Can we just export it?

Cannot unmarshal object into Go struct

Hello, have an issue on GetDashboardByUID method. On some dashboards getting errors such as

2021/03/05 13:27:41 unmarshal board: json: cannot unmarshal object into Go struct field TemplateVar.templating.list.query of type string for db/kubernetes-compute-resources-workload
2021/03/05 13:27:42 unmarshal board: json: cannot unmarshal number into Go struct field Board.panels of type string for db/kubernetes-networking-namespace-pods
2021/03/05 13:27:42 unmarshal board: json: cannot unmarshal number into Go struct field Board.panels of type string for db/kubernetes-networking-pod
2021/03/05 13:27:43 unmarshal board: json: cannot unmarshal object into Go struct field TemplateVar.templating.list.query of type string for db/loki-quick-search

And this

2021/03/05 15:48:17 unmarshal board: json: cannot unmarshal number into Go struct field Board.panels of type string for db/auth
2021/03/05 15:48:20 unmarshal board: json: cannot unmarshal object into Go struct field Board.panels of type []struct { Type string "json:\"type,omitempty\""; Params []string "json:\"params,omitempty\"" } for db/aws-cluster
2021/03/05 15:48:30 unmarshal board: json: cannot unmarshal number into Go struct field Board.panels of type string for db/production

Additional details here

Repeated serialization de-serialization causes strange nesting of CustomPanel in serialized JSON representation

Hi Folks,

We are building a piece of software that requires pulling dashboards, modifying them programmatically and doing so repeatedly on certain user inputs.

In our first attempt to prototype it we discovered that serializing and de-serializing JSON dashboard content to-from sdk.Board causes strange nesting of CustomPanel (assuming there is such data).

I wrote a test to highlight the problem as shown below and wondering if there are ways to avoid such nesting in the serialized JSON output. Thank you for your help.

deployed version:

└─ $ ▶ helm list
NAME              	NAMESPACE 	REVISION	UPDATED                                	STATUS  	CHART                  	APP VERSION     
grafana-xyz-system	xyz-system	2       	2021-03-30 08:03:34.585596381 -0700 PDT	deployed	grafana-6.4.8          	7.4.2           

lib version:

└─ $ ▶ cat go.mod | grep grafana
	github.com/grafana-tools/sdk v0.0.0-20201123153837-5fb28a7aa2ef

The test:

package grafanatest

import (
	"encoding/json"
	"fmt"
	"github.com/grafana-tools/sdk"
	"testing"
)

// TestCustomPanelSerialization serializes and de-serializes a sample dashboard
// object a few times and prints out final serialized JSON data to highlight
// that CustomPanel is nested in a strange way.
func TestCustomPanelSerialization(t *testing.T) {
	board := &sdk.Board{
		ID:              0,
		UID:             "",
		Slug:            "",
		Title:           "",
		OriginalTitle:   "",
		Tags:            nil,
		Style:           "",
		Timezone:        "",
		Editable:        false,
		HideControls:    false,
		SharedCrosshair: false,
		Panels: []*sdk.Panel{
			{
				CommonPanel:     sdk.CommonPanel{},
				GraphPanel:      nil,
				TablePanel:      nil,
				TextPanel:       nil,
				SinglestatPanel: nil,
				DashlistPanel:   nil,
				PluginlistPanel: nil,
				RowPanel:        nil,
				AlertlistPanel:  nil,
				CustomPanel: &sdk.CustomPanel{
					"key": "value",
							},
						},
		},
		Rows:       nil,
		Templating: sdk.Templating{},
		Annotations: struct {
			List []sdk.Annotation `json:"list"`
		}{},
		Refresh:       nil,
		SchemaVersion: 0,
		Version:       0,
		Links:         nil,
		Time:          sdk.Time{},
		Timepicker:    sdk.Timepicker{},
		GraphTooltip:  0,
	}

	jb, err := json.Marshal(board)
	if err != nil {
		t.Fatal(err)
	}

	for i := 0; i < 10; i++ {
		newBoard := &sdk.Board{}
		if err := json.Unmarshal(jb, newBoard); err != nil {
			t.Fatal(err)
		}

		jb, err = json.MarshalIndent(newBoard, "", "  ")
		if err != nil {
			t.Fatal(err)
		}
	}

	fmt.Println(string(jb))
}

the output of this is as follows... pl. see nesting of CustomPanel:

{
  "slug": "",
  "title": "",
  "originalTitle": "",
  "tags": null,
  "style": "",
  "timezone": "",
  "editable": false,
  "hideControls": false,
  "sharedCrosshair": false,
  "panels": [
    {
      "editable": false,
      "error": false,
      "gridPos": {},
      "id": 0,
      "isNew": false,
      "span": 0,
      "title": "",
      "transparent": false,
      "type": "",
      "CustomPanel": {
        "CustomPanel": {
          "CustomPanel": {
            "CustomPanel": {
              "CustomPanel": {
                "CustomPanel": {
                  "CustomPanel": {
                    "CustomPanel": {
                      "CustomPanel": {
                        "CustomPanel": {
                          "CustomPanel": {
                            "key": "value"
                          },
                          "editable": false,
                          "error": false,
                          "gridPos": {},
                          "id": 0,
                          "isNew": false,
                          "span": 0,
                          "title": "",
                          "transparent": false,
                          "type": ""
                        },
                        "editable": false,
                        "error": false,
                        "gridPos": {},
                        "id": 0,
                        "isNew": false,
                        "span": 0,
                        "title": "",
                        "transparent": false,
                        "type": ""
                      },
                      "editable": false,
                      "error": false,
                      "gridPos": {},
                      "id": 0,
                      "isNew": false,
                      "span": 0,
                      "title": "",
                      "transparent": false,
                      "type": ""
                    },
                    "editable": false,
                    "error": false,
                    "gridPos": {},
                    "id": 0,
                    "isNew": false,
                    "span": 0,
                    "title": "",
                    "transparent": false,
                    "type": ""
                  },
                  "editable": false,
                  "error": false,
                  "gridPos": {},
                  "id": 0,
                  "isNew": false,
                  "span": 0,
                  "title": "",
                  "transparent": false,
                  "type": ""
                },
                "editable": false,
                "error": false,
                "gridPos": {},
                "id": 0,
                "isNew": false,
                "span": 0,
                "title": "",
                "transparent": false,
                "type": ""
              },
              "editable": false,
              "error": false,
              "gridPos": {},
              "id": 0,
              "isNew": false,
              "span": 0,
              "title": "",
              "transparent": false,
              "type": ""
            },
            "editable": false,
            "error": false,
            "gridPos": {},
            "id": 0,
            "isNew": false,
            "span": 0,
            "title": "",
            "transparent": false,
            "type": ""
          },
          "editable": false,
          "error": false,
          "gridPos": {},
          "id": 0,
          "isNew": false,
          "span": 0,
          "title": "",
          "transparent": false,
          "type": ""
        },
        "editable": false,
        "error": false,
        "gridPos": {},
        "id": 0,
        "isNew": false,
        "span": 0,
        "title": "",
        "transparent": false,
        "type": ""
      }
    }
  ],
  "rows": null,
  "templating": {
    "list": null
  },
  "annotations": {
    "list": null
  },
  "schemaVersion": 0,
  "version": 0,
  "links": null,
  "time": {
    "from": "",
    "to": ""
  },
  "timepicker": {
    "refresh_intervals": null,
    "time_options": null
  }
}

UserListing Discrepancy

GET /api/users uses the flag "isAdmin" to flag the user as an Admin.
GET /api/users/:id uses the flag "isGrafanaAdmin" to flag the user as an Admin.

Since both responses in the API use the same sdk.Model, the behavior is inconsistent and is reporting the wrong value.

specifically, client.GetAllUsers will never return IsGrafanaAdmin true.

Inconsistent Grafana Behavior

I think this is an issue with grafana itself but I wanted to call this out.

Let's take this panel as an example truncated.

{
            "datasource": "Flow - All Traffic",
            "description": "",
            "targets": [
                {
                    "aggregation": "Last",
                    "alias": "",
                    "bucketAggs": [
                        {
                            "$$hashKey": "object:206",
                            "fake": true,
                            "field": "meta.router.name.keyword",
                            "id": "3",
                            "settings": {
                                "min_doc_count": "1",
                                "order": "desc",
                                "orderBy": "1",
                                "size": "6"
                            },
                            "type": "terms"
                        },
                        {
                            "$$hashKey": "object:207",
                            "field": "start",
                            "id": "2",
                            "settings": {
                                "interval": "auto",
                                **"min_doc_count": 0,**
                                **"min_doc_count": "1",**
                                "trimEdges": 0
                            },
                            "type": "date_histogram"
                        }
                    ],

Notice how once min_doc_count is set to a numeric value of 0 and another time it's a quoted string of "1". The SDK is breaking as expected because min_doc_count is defined as an in the panel. So I can dump the raw data but I can't load the JSON back as a Board in order to export to grafana.

Any thoughts on how to address this?

Is PR welcomed?

I am working on a project which needs to interact with grafana. While this seems the official repo for the sdk, it lacks a lot of functionalities I need. If PR welcomed, I can contribute back the functionalities I found useful and add it back here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.