Giter VIP home page Giter VIP logo

pgollangi / firestore-grafana-datasource Goto Github PK

View Code? Open in Web Editor NEW
10.0 3.0 3.0 2.36 MB

Google Firestore data source plugin for Grafana

Home Page: https://grafana.com/grafana/plugins/pgollangi-firestore-datasource/

License: MIT License

JavaScript 9.95% Dockerfile 1.80% TypeScript 38.03% Go 50.21%
grafana firestore firestore-database gcp-firestore google-firestore grafana-datasource grafana-plugin

firestore-grafana-datasource's Introduction

CI Release

firestore-grafana-datasource

Google Firestore Data Source Plugin for Grafana.

Grafana Firestore Data Source Plugin enables integrating data on Firestore into Grafana dashboards.

It uses FireQL to capture user query that translated to issue queries on Firestore and construct results.

FireQL is a Go library to query Google Firestore database using SQL syntax.

Features

  • Use Google Firestore as a data source for Grafana dashboards
  • Configure Firestore data source with GCP Project Id and Service Account for authentication
  • Store Service Account data source configuration in Grafana encrypted storage Secure JSON Data
  • Query Firestore collections and path to collections
  • Auto detect data types: string, number, boolean, json, time.Time
  • Query selected fields from the collection
  • Order query results
  • Limit query results
  • Query Collection Groups
  • Count query results
  • Use of Grafafa global variables in queries.

Firestore data source configuration

Using datasource

Getting started

Frontend

  1. Install dependencies

    yarn install
  2. Build plugin in development mode or run in watch mode

    yarn dev
    
    # or
    
    yarn watch
  3. Build plugin in production mode

    yarn build
  4. Run the tests (using Jest)

    # Runs the tests and watches for changes
    yarn test
    
    # Exists after running all the tests
    yarn lint:ci
  5. Spin up a Grafana instance and run the plugin inside it (using Docker)

    yarn server
  6. Run the E2E tests (using Cypress)

    # Spin up a Grafana instance first that we tests against 
    yarn server
    
    # Start the tests
    yarn e2e
  7. Run the linter

    yarn lint
    
    # or
    
    yarn lint:fix

Backend

  1. Update Grafana plugin SDK for Go dependency to the latest minor version:

    go get -u github.com/grafana/grafana-plugin-sdk-go
    go mod tidy
  2. Build backend plugin binaries for Linux, Windows and Darwin:

    mage -v
  3. List all available Mage targets for additional commands:

    mage -l

Push a version tag

To trigger the workflow we need to push a version tag to github. This can be achieved with the following steps:

  1. Run npm version <major|minor|patch>
  2. Run git push origin main --follow-tags

Contributing

Thanks for considering contributing to this project!

Please read the Contributions and Code of conduct.

Feel free to open an issue or submit a pull request!

License

MIT

firestore-grafana-datasource's People

Contributors

pgollangi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

firestore-grafana-datasource's Issues

Grafana template variable support

Describe the solution you'd like
Right now the plugin doesn't support Grafana template variables. It would be really useful to have this feature, because right now, I have to create separate dashboards for collections. A variables which queries the collection names, or even "Custom" type variable support for queries would be great to have.

Additional context
image

Will there be support for version 10?

These days I wanted to connect the plugin to Grafana but I have not been able to because it has been updated to version 10. I would like to know if there will be support for this version in the near future. Thank you very much!

Connect to local Firestore Emulator

First of all: thank you for building such a cool plugin!

I'm developing an app using the local Firebase eumlators, including the local Firestore emulator. I want to use Grafana to query some documents on that local Firestore DB. The problem: when configuring this plugin, I don't see an option to tell it to connect to the local Firestore instance. I can only configure a name, projectId and the service account details.

Ideally, I could "flip a switch" and tell the plugin to connect to the local instance using a local URL like
"http://localhost:8080"

Thanks
Kurt

Error when requesting data

Describe the bug
I am trying to run the following query, select * from orders, in order to get the corresponding data but it does not work and in the query inspector it says the following:

{
"request": {...},
"response": {
"message": "An error occurred within the plugin",
"messageId": "plugin.downstreamError",
"statusCode": { "statusCode": 500,
"traceID": "56cf6ca5699814f9"
}
}

I'm currently running the same query but with another table and it's working normally. Does anyone know what it could be?

Expected behavior
The expected result would be to be able to fetch the information from the table.

Screenshots
image
image

I am using grafana cloud version 9.5.3. From the grafana website.

accessing subcollections

in my database i have a collection of users and then for each user a subcollection.
is there a documentation on how to access the subcollections and the data because everything i have tried it returns an error on the syntax.
I need to loop for each document id and get the data

Add option to connect to named firestore database

Is your feature request related to a problem? Please describe.
I'm running multiple managed Firestore databases in GCP, so I can not use the default

Describe the solution you'd like
When creating the datasource, an optional input field for database id

Describe alternatives you've considered
Right now looking into writing a separate service to subscribe to a collection and write it to an InfluxDB instead

Additional context
The go firestore library has support for selecting database by using firestore.NewClientWithDatabase(ctx, project, db)

Grafana 10 support

Hello ๐Ÿ‘‹

Just letting you know that Grafana 10 is coming out soon and your grafanaDependency is currently set to not support it. This means customers that will get Grafana 10 will not be able to use your plugin.

You can test your plugin compatibility using the latest Grafana 10 grafana/grafana-dev docker images

"grafanaDependency": "^9.2.5",

Getting plugin unavailable in query inspector

Describe the bug
Grafana v9.3.2, running on an Ubuntu AWS instance

  • Installed the plugin via Grafana CE UI.
  • Successfully configured Firestore data source using GCP service account credentials
  • Created new test Dashboard using the new Firestore data source
  • Created new panel, selected new Firestore datasource
  • Panel shows no data.
  • Query inspector shows the following in the json response
Object
request:Object
url:"api/ds/query"
method:"POST"
data:Object
queries:Array[1]
range:Object
from:"1641024000000"
to:"1672559999999"
hideFromInspector:false
response:Object
message:"An error occurred within the plugin"
messageId:"plugin.downstreamError"
statusCode:500
traceID:""

OS log shows the following (Grafana log level = debug)

logger=secrets.kvstore t=2023-02-21T00:11:50.147991679Z level=debug msg="got secret value" orgId=1 type=datasource namespace=Firestore-backend
logger=plugin.pgollangi-firestore-datasource t=2023-02-21T00:11:50.150593661Z level=debug msg="QueryData called" numQueries=1
logger=plugin.pgollangi-firestore-datasource t=2023-02-21T00:11:50.150980051Z level=debug msg="FirestoreQuery: " EXTRA_VALUE_AT_END="map[CollectionPath:user_events IsCount:false Limit:0 OrderBy:<nil> Select:<nil> Where:<nil>]"
logger=plugin.pgollangi-firestore-datasource t=2023-02-21T00:11:50.151356351Z level=debug msg="Query ready!"
logger=plugin.pgollangi-firestore-datasource t=2023-02-21T00:11:50.402701399Z level=debug msg="panic: interface conversion: interface {} is []float64, not []string"
logger=plugin.pgollangi-firestore-datasource t=2023-02-21T00:11:50.403348915Z level=debug msg=
logger=plugin.pgollangi-firestore-datasource t=2023-02-21T00:11:50.403789611Z level=debug msg="goroutine 33 [running]:"
logger=plugin.pgollangi-firestore-datasource t=2023-02-21T00:11:50.404847291Z level=debug msg="github.com/pgollangi/firestore/pkg/plugin.(*Datasource).query(0xc000126c00?, {0xea3578, 0xc00045c270}, {0x1, {0xc00026e7a0, 0x1e}, 0xc000466200, 0x0, 0xc000126c00}, {{0x13f0448, ...}, ...})"
logger=plugin.pgollangi-firestore-datasource t=2023-02-21T00:11:50.40526078Z level=debug msg="\t/home/runner/work/firestore-grafana-datasource/firestore-grafana-datasource/pkg/plugin/datasource.go:189 +0x1525"
logger=plugin.pgollangi-firestore-datasource t=2023-02-21T00:11:50.40580811Z level=debug msg="github.com/pgollangi/firestore/pkg/plugin.(*Datasource).QueryData(0xc00027d580?, {0xea3578, 0xc00045c270}, 0xc000275cc0)"
logger=plugin.pgollangi-firestore-datasource t=2023-02-21T00:11:50.406305742Z level=debug msg="\t/home/runner/work/firestore-grafana-datasource/firestore-grafana-datasource/pkg/plugin/datasource.go:61 +0x276"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.