datalens-tech / datalens Goto Github PK
View Code? Open in Web Editor NEWA modern, scalable analytics system
Home Page: https://datalens.tech
License: Apache License 2.0
A modern, scalable analytics system
Home Page: https://datalens.tech
License: Apache License 2.0
Enable BigQuery connector
I have PostgreSQL and Clickhouse databases running on a server. Both are accessible via DBeaver or via http
But when I try to make a connection in Datalens, connection attempt fails, with the error like this(for clickhouse):
Data source refused connection. ERR.DS_API.DB.SOURCE_CONNECT_ERROR { "data": { "details": { "db_message": "HTTPSConnectionPool(host='192.168.1.224', port=8123): Max retries exceeded with url: /?query_id=e7c0bdde-b9f7-11ee-bdd0-0242ac120006&database=system&join_use_nulls=1&distributed_ddl_task_timeout=280&send_progress_in_http_headers=0&output_format_json_quote_denormals=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f43f6287130>, 'Connection to 192.168.1.224 timed out. (connect timeout=1.0)'))" }, "message": "Data source refused connection.", "debug": { "db_message": "HTTPSConnectionPool(host='192.168.1.224', port=8123): Max retries exceeded with url: /?query_id=e7c0bdde-b9f7-11ee-bdd0-0242ac120006&database=system&join_use_nulls=1&distributed_ddl_task_timeout=280&send_progress_in_http_headers=0&output_format_json_quote_denormals=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f43f6287130>, 'Connection to 192.168.1.224 timed out. (connect timeout=1.0)'))", "query": "select 1" }, "code": "ERR.DS_API.DB.SOURCE_CONNECT_ERROR" } }
The same problem appears while running Datalens container on the server or on the other machine in network.
Could you please tell me, what may be the issue?
Will it be added in the future?
Hello!
I decided to scan docker images, that have been creted after docker-compose up -d command. So, Snyk've found multiple vulnerabilities in datalens images. In attachment you can find full scan results. The great concern is about critical vulnerabilities with RCE. Are they affect to current version of datalnes? Is there any plans to update old components in the future?
Thank you!
datalens-us_0.96.0.txt
datalens-ui_0.795.0.txt
datalens-data-api_0.2037.0.txt
datalens-control-api_0.2037.0.txt
We are ready to test your product. Our analysts noticed that there is no possibility of uploading to Excel. How to fix it ?
In cloud version MS SQL is available, but not in open source. We have MS SQL in private network - the only way to connect is to use DataLens self-hosted.
It would be great if MS SQL connector become public.
Great thanks to your team.
добрый день, была бы полезна функциональность печати дашбоардов и экспорта как pdf
Everything needed to enable file connector as a part of DataLens OSS is present across repositories.
Steps required:
After installation I can't create connections. When I click "Create connection" I only see a white sheet.
In the cmopose file all parameters are standard.
Hi, datalens community!
Is it possible to make a custom Datalens plugin chart?
Any guides or information will be appreciated.
Best regards!
After executing command - docker compose down&&docker compose up -d
I am loosing all my settings. I tried to attach volumes to containers pg-compeng, pg-us
like this:
volumes:
- pg-compeng:/var/lib/postgresql/data
volumes:
- pg-us:/var/lib/postgresql/data
volumes:
pg-compeng:
pg-us:
Can anyone help me make the docker-compose file resistant to reboots?
Will you provide Chinese translation ? Or how could I make a pull request on translation
Please add export/import configurations for Dashboard/Chart/Connections all this stuff.. in some file format (JSON/YAML/...), like grafana does.
Hey there,
I’d love to test DataLens with Snowflake. Any plans adding snowflake support?
I defined a dataset on a table with 21 billion records.
When I tried to create my first chart, it did this:
![Screenshot_20230926_190229](https://github.com/datalens-tech/datalens/assets/18581488/d81e4044-9b75-4eff-8107-98b6f15dd4ce)
Running this query on ClickHouse:
SELECT `t1`.`created_at` AS `res_0` FROM `default`.`twitter` AS `t1` GROUP BY `res_0` LIMIT 1000001
The query will process nicely, but the aggregation is performed by every unique time point, and it requires 94.5 seconds to finish on a service in ClickHouse Cloud.
Guys, a very big thanks for sharing DataLens! It would be nice to have map-charts in open source version.
Please advice how to eneble XLSX export option. CSV export not working well also. Any selected encoding shows incorrect results when open in Excel.
Please add instructions for backing up and restoring datalens configuration data (connections, datasets, charts, dashboards). For example, I want to move datalens to a new server. Please also add update instructions. I want to update DataLens to a new version and am afraid of losing my data.
I am facing issue while connecting my database(PostgreSQL). I am running this (datalens project) on my wsl2. How can I able to connect my PostgreSQL database to it ? Should I run PostgreSQL server on wsl2 as well? Or I can connect this via remote connection to my Windows machine already running PostgreSQL server on it.
I've tried both and getting errors, still unable to connect to DB :(
Enable Bitrix24 connector
Prerequisites:
ECONNABORTED
{
"title": "Ошибка",
"description": "Превышено время ожидания ответа"
}
[paulmer@docker-01 ~]$ cat /etc/os-release
NAME="Fedora Linux"
VERSION="38 (Thirty Eight)"
ID=fedora
VERSION_ID=38
VERSION_CODENAME=""
PLATFORM_ID="platform:f38"
PRETTY_NAME="Fedora Linux 38 (Thirty Eight)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:38"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f38/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=38
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=38
SUPPORT_END=2024-05-14
[paulmer@docker-01 ~]$
C:\Windows\system32>git clone https://github.com/datalens-tech/datalens
Cloning into 'datalens'...
remote: Enumerating objects: 35, done.
remote: Counting objects: 100% (35/35), done.
remote: Compressing objects: 100% (27/27), done.Receiving objects: 100% (35/35), 13.48 KiB | 299.00 KiB/s, done.
Resolving deltas: 100% (15/15), done.C:\Windows\system32>cd datalens
C:\Windows\System32\datalens>set HC=1
C:\Windows\System32\datalens>docker compose up
[+] Running 1/0
✔ Network datalens_default Created 0.0s
- Container datalens-pg-us-1 Creating 0.0s
- Container datalens-pg-compeng-1 Creating 0.0s
- Container datalens-postgres-connection-1 Creating 0.0s
Error response from daemon: mkdir C:\Windows\System32\datalens\us-data: Access is denied.
When trying to connect to metadata PostgreSQL DB with TLS/SSL, the following error appears when running scripts/preflight.sh
(I was trying to connect to Yandex Managed Service for PostgreSQL):
error: odyssey: c3577ff1eb926: SSL is required
Although the correct root certificate is present in /certs/root.crt
, it does not help.
It seems like here certificate path is not passed to initPosgresDB()
(although I could not find the signature of this function since gravity-ui/postgreskit
is probably a private repo).
The solution that worked for me was to add the following parameters to DSN:
postgres://{user}:{password}@{host}:{port}/{database}?sslmode=verify-full&sslrootcert=/certs/root.crt
.
I think that this solution may be added to README.
Broken demo «Интерактивный пример встроенного чарта» in a documentation post
https://cloud.yandex.ru/blog/posts/2021/06/datalens-digest-june-2021
Hello there.
Do you have plans to add Trino connector or custom JDBC connector for your BI-system?
In Power BI, it is possible to create report templates to use as a basis for other reports.
I would like to see similar functionality in DataLens.
https://learn.microsoft.com/en-us/power-bi/create-reports/desktop-templates
Currently our containers are running entry-point container processes using root user, which is far from ideal. We should switch to non-privileged users in all our containers:
This non-root users should have minimum possible access to the files in the container.
Enable Greenplum connector
I see that you're recently added Oracle connector, as well as some other ones. Would be nice to have MySQL connector in open-source version.
Thanks in advance!
When connecting to Yandex Cloud Managed Service for PostgreSQL, it gives error. DataLens is deployed locally, loading a certificate file via Advanced connection settings does not help.
Through dbeaver I can connect to the database from this machine.
OS: Linux version 6.2.0-34-generic (buildd@lcy02-amd64-025) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~23.04) 12.3.0, GNU ld (GNU Binutils for Ubuntu) 2.40)
CPU: AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx
Error text:
Database error. ERR.DS_API.DB { { "data": { { "code": "ERR.DS_API.DB", "details": { "db_message": "connection to server at "..." (...), port 6432 failed: could not open certificate file \"/root/.postgresql/postgresql.crt\": Permission denied\n"" }, "message": "Database error." "debug": { "db_message": "connection to server at "..." (...), port 6432 failed: could not open certificate file \"/root/.postgresql/postgresql.crt\": Permission denied\n". "query": "select 1" } } }
Hello there.
Could you please enable ClickHouse connector option for conecction without user and password? It's now strongly required.
I'm following the readme on the Ubuntu machine, and for some reason, I have to type docker-compose
instead of docker compose
.
After Windows 10 Update KB4074588, some ports are reserved by Windows and applications cannot bind to these ports. 50500 is in the blocked range.
You can use netsh interface ipv4 show excludedportrange protocol=tcp to list the reserved ranges.
When I create a new connection to the "Managed Service for PostgreSQL" database for use in collections, I select the root certificate; it is not used for the connection. The following error appears.
connection to server at "***********.mdb.yandexcloud.net" (158.XX.XX.XX), port 6432 failed: could not open certificate file "/root/.postgresql/postgresql.crt": Permission denied
Despite the fact that the US service is connected to the same database using a similar connection string METADATA_POSTGRES_DSN_LIST="postgres://{user}:{password}@{host}:{port}/{database}?sslmode=verify-full&sslrootcert=/certs/ root.crt"
I tried to upload the root certificate into the container, changed the rights to the root directory (for testing), this helps to find the certificate, but then I get another error:
connection to server at "***********.mdb.yandexcloud.net" (158.XX.XX.XX), port 6432 failed: could not open certificate file "/root/.postgresql/postgresql.key"
Hi there!
Are there any docs (besides those that are in Yandex.Cloud docs) covering the creation of datasets and possible error codes? I'm trying to create a dataset with an SQL query for a connected Postgres connection, but I'm stuck with this:
Relation requires at least one condition
ERR.DS_API.FORMULA
{
"title": "ERR.DS_API.FORMULA",
"description": "Relation requires at least one condition"
}
ERR.DS_API.VALIDATION.ERROR
[
{
"id": "340a4261-635d-11ee-85cb-db624a6af76d",
"type": "avatar_relation",
"errors": [
{
"level": "error",
"code": "ERR.DS_API",
"message": "Relation requires at least one condition",
"details": {}
}
]
}
]
Unfortunately, nothing was found on the Internet, except this page, but it doesn't help :(
The query works independently and it has conditions.
Thanks!
Hello!
Please give info how to add mssql connection.
Thanks!
уберите, пожалуйста, запуск образов из под root
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=ru
We want to create connector from DL to Datahub, please create API endpoint to get metadata from DL.
C:\Users\spull\ideaprojects\datalens>docker compose up
[+] Running 4/4
✘ control-api Error 0.6s
✘ datalens Error 0.6s
✘ data-api Error 0.6s
✘ us Error 0.6s
Error response from daemon: Head "https://ghcr.io/v2/datalens-tech/datalens-data-api/manifests/0.2037.0": denied: denied
Enable Metrica/AppMetrica connectors
Open questions:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.