tellery / tellery Goto Github PK

When multiple people execute the same SQL at the approach time, the backend should not send all of them to the connector. But only once then return the result to all requests.

Describe the solution you'd like

• Calculate the Hash of SQL needs to be executed
• Place Hash in a global storage ( pg or redis )
• Before performing a SQL, first check the store to confirm if the SQL is currently being executed.
  • If the SQL is not being executed, submit it to the Connector and store it
  • If the SQL is being executed, waiting until the previous SQL is completed and return the result

Additional context

We need to take into account the expansion of Backend. Thus we need to save the results of SQL to global storage. Let the waiting request can get this result.

Maybe we can set an Index for each Pending SQL. When there is a executing sql request, make the Pending SQL's index plus 1. When the Pending request gets SQL Result, make the Pending SQL's index sub 1.

If the Index is 0, delete the Pending SQL Record.

RIP Tellery

Really like the amazing editor you built (Notion-like editor). Wondering if there's a way to run just the editor alone? For example, as an npm package that has the self contained editor.

Describe the bug

The test case "sort user members" has a probability of failure

Link

https://github.com/tellery/tellery/runs/3250998783?check_suite_focus=true

When user mouses over a data asset in the left sidebar, display its detailed information.

Creating query from the left sidebar will not create a associated visualization block in the current story

• Create new query
• Import from other story
• Import a CSV file

All the queries listed in the left sidebar can be deleted/duplicated by the users who has write access to the story.

Describe the bug

The test case "test active users in story" has a probability of failure

Link

https://github.com/tellery/tellery/runs/3250892367?check_suite_focus=true

Is your feature request related to a problem? Please describe.
We base on some principles to refactor connector server:

1. Profile system is base on the file system, it is unsuitable to be handle in multiply pods.
2. In the foreseeable future, some modules will provide two implements for on-prem and SaaS, the current code architecture can't handle the feature very well.
3. It's hard for community users to contribute code about integration or create a new integration.

Describe the solution you'd like

1. We provide a common interface and two implements:

• file system implement just for the on-prem users.
• database implement for cluster users.

2. Import an injection framework to solve the problem, Koin may be the most succinct choice, and then update some managers and services to fit the injection framework.
3. Refactor integration module, decoupling it with profile module.

Describe the bug
see the issue title

To Reproduce

1. Go to https://demo.tellery.io
2. Click Command+k
3. Input te
4. See error

Screenshots

Desktop (please complete the following information):

• Browser [chrome]
• Version [online demo]

Describe the bug
After following the steps mentioned in GitHub to deploy Tellery locally, I am getting UnauthorizedError: not exist error every time even after clean installtion.

tellery-server_1 | <-- GET /
tellery-server_1 | xxx GET / 401 33ms -
tellery-server_1 | UnauthorizedError: not exist
tellery-server_1 | at Function.notExist (/tellery/dist/src/error/error.js:46:16)
tellery-server_1 | at UserService.verifyToken (/tellery/dist/src/services/user.js:154:45)
tellery-server_1 | at processTicksAndRejections (node:internal/process/task_queues:96:5)
tellery-server_1 | at async user (/tellery/dist/src/middlewares/user.js:23:23)
tellery-server_1 | at async logger (/tellery/node_modules/koa-logger/index.js:67:7)
tellery-server_1 | at async error (/tellery/dist/src/middlewares/error.js:5:9) {
tellery-server_1 | status: 401
tellery-server_1 | }
tellery-server_1 | <-- GET /favicon.ico
tellery-server_1 | xxx GET /favicon.ico 401 6ms -
tellery-server_1 | UnauthorizedError: not exist
tellery-server_1 | at Function.notExist (/tellery/dist/src/error/error.js:46:16)
tellery-server_1 | at UserService.verifyToken (/tellery/dist/src/services/user.js:154:45)
tellery-server_1 | at processTicksAndRejections (node:internal/process/task_queues:96:5)
tellery-server_1 | at async user (/tellery/dist/src/middlewares/user.js:23:23)
tellery-server_1 | at async logger (/tellery/node_modules/koa-logger/index.js:67:7)
tellery-server_1 | at async error (/tellery/dist/src/middlewares/error.js:5:9) {
tellery-server_1 | status: 401

And if I go to http://localhost:8000/ I get {"errMsg":"not exist"}

To Reproduce
Steps to reproduce the behavior:

1. Go to installation directory and docker-compose up.
2. See error

Expected behavior
Expect to run on local server smoothly

Screenshots

Desktop (please complete the following information):

• OS: MacOS 11.5.2 (Intel i7)
• Browser: Chrome
• Version: 92

I would like to make life easier for Tellery maintainers by implementing Uffizzi preview environments.
Disclaimer: I work on Uffizzi.

Uffizzi is a Open Source full stack previews engine and our platform is available completely free for Tellery (and all open source projects). This will provide maintainers with preview environments of their PRs in the cloud, allowing them iterate faster and reduce time to merge.

Uffizzi is purpose-built for the task of previewing PRs and it integrates with your workflow to deploy preview environments in the background without any manual steps for maintainers or contributors.

TODO:

• Intial PoC

Is your feature request related to a problem? Please describe.

We should allow developers to add a custom Block in Tellery by writing certain functions.

Describe the solution you'd like

This feature can be implemented by defining an Interface which call BlockExtensionInterface(BEI).

What is the BEI like

BlockExtensionInterface interface {
  
}

Additional context

Now there is a little doubt that whether the ext blocks can be loaded dynamically. If we need to achieve this future, there are two ways.

1. Start a process for the BEI plugin, communicate by RPC or other means
2. Start a VM in Nodejs to dynamically execute the BEI code

If the first method is used, the management difficulty of service will rise. If you use the second method, this means that all backend services must share a store to ensure that the same BEI code is obtained.

TODO List

• Define BEI
• Decoupled the relationship betwen QuestionBlocks and SQL Execution

Drag-and-drop a data asset from the left sidebar to story:

• A query will be generated to explore the configured metrics of the data asset
• An exeuction will be trigger against the query to fetch results
• A visualization block will be created and connected to that query

Describe the bug

When using pg, the cache cannot be shared when there are multiple backend instances.

As a result, online users cannot be aggregated and counted in socket services.

Additional context

The reason for this problem is that no external cache is used when using node-cache-manager.

ps: there is currently no implementation of the node-cache-manager with postgresql.

Is your feature request related to a problem? Please describe.
Storing profile as a json file is a fault of early design since in the very beginning we didn't consider modifying profiles from web UI. Currently, we met the following issues:

Issue 1: mount profile by configMap in k8s => not be able to update the profile from web UI, since the file is read-only.

Issue 2: It's hard to handle the case of multiple instances writing into a single file.

Describe the solution you'd like
The connector should connect to a Postgres to store profile, and a redis as an event bus in HA usage (this could opt-out)

For simple deployment (i.e., without HA), the Postgres can be replaced by a sqlite maybe?

A metric must be associated with a column of the data asset.

A metric contains two parts:

• An aggregation function used to calculate the metric, like sum,avg
• A name for description purpose

In normal mode, the cookie in the browser is cleared when the user logs out.
In anonymous mode, users cannot actively log out.

The current approach is to call the isAnonymous function directly in router.ts to determine whether it is in anonymous mode.

But in fact, we have implemented AnonymousUserService, and we can implement the logic of logout in UserService and AnonymousUserService.

You can implement the logout method in UserService and return a boolean to determine whether the cookie is clear or not

Describe the bug
After deploying the service locally and logged in, the first block of the first story shows "deleted block"

To Reproduce
Steps to reproduce the behavior:

1. deploy the server by docker-compose
2. log in
3. go to the first story

Expected behavior
Appears the visualization of Iris data set

Hi, I was curious what editor was used for text editing (like DraftJS, CKEditor, or Prosemirror etc.) but couldn't find anything that would give me the clue. Was the Tellery editor write from scratch?

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Describe the bug
获取镜像失败

To Reproduce
Steps to reproduce the behavior:
1、使用 docker 安装案例，进行到docker-compose up时无法拉取镜像
`% docker-compose up

WARNING: The EMIAL_PORT variable is not set. Defaulting to a blank string.
WARNING: The EMIAL_HOST variable is not set. Defaulting to a blank string.
WARNING: The EMIAL_FROM variable is not set. Defaulting to a blank string.
Pulling redis (redis:6.2.4)...
ERROR: Head https://registry-1.docker.io/v2/library/redis/manifests/6.2.4: Get https://auth.docker.io/token?scope=repository%3Alibrary%2Fredis%3Apull&service=registry.docker.io: EOF`

Expected behavior

Screenshots

Desktop (please complete the following information):

• OS: [e.g. MacOS ]

Additional context
Add any other context about the problem here.

Contribution guidelines are urgently needed to guide developers to do the following, for example

• How do I run the entire service locally
• How to contribute code, build, debug and test