Giter VIP home page Giter VIP logo

gridstudio's Introduction

Grid studio is a web-based spreadsheet application with full integration of the Python programming language.

main-video.mp4

It intends to provide an integrated workflow for loading, cleaning, manipulating, and visualizing data. This is achieved through a spreadsheet backend written in Go with integration of the Python runtime to manipulate its contents.

Architecture overview

The application is structured in two parts:

  1. The (centralized) workspace manager
    1. CRUD interface for creating, copying, editing and deleting workspaces.
    2. Proxy to send traffic to the right workspace environment (part 2)
  2. Workspace Go execution environment
    1. Go cell parsing and evaluating spreadsheet backend
    2. Node.js terminal session
    3. Python interpreter integration

For more details about each part check out the code in the repository. If anything is unclear (or unreadable - not all code is equally pretty!) make an issue and details will be provided.

Features

Spreadsheet functions that you know

feature-spreadsheet.mp4

Powerful scripting, fully integrated

feature-python.mp4

Run any command on Ubuntu Linux

feature-terminal.mp4

Installation

To run Grid studio locally refer to the Installation page of the Wiki.

It comes down to pulling the latest Grid studio Docker image that has all dependencies configured (mainly: Go language, Python 3 with packages, Node.js) and starting the Docker container.

For more information check out our Wiki.

gridstudio's People

Contributors

0xflotus avatar 1kastner avatar allenfoxtc avatar dependabot[bot] avatar eiso avatar gliptak avatar iambocai avatar ibadeecodes avatar montudor avatar ricklamers avatar yassineaddi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gridstudio's Issues

Review code execution UX

Code execution is unintuitive. It seems like if nothing is selected, it executes by stepping line by line. The first time I tried it, my insertion point was underneath my code, so nothing executed. And then to define a function, you have to select the entire thing before executing.

Contributed by: rcfox

sed errors

Just tried running run.sh and ran into this error. Seems like the docker pull is continuing though.

Screen Shot 2019-08-01 at 9 31 41 AM

sed: illegal option -- r
usage: sed script [-Ealn] [-i extension] [file ...]
       sed [-Ealn] [-i extension] [-e script] ... [-f script_file] ... [file ...]
sed: illegal option -- r
usage: sed script [-Ealn] [-i extension] [file ...]
       sed [-Ealn] [-i extension] [-e script] ... [-f script_file] ... [file ...]

Allow naming of column/row heads

This would allow one to use more natural nomenclature, along the lines of Lotus Improv, Quantrix, and the opensource (but abandoned) Flexisheet.

Unable to install on windows 10

$ ./run.sh
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: error during connect: Post http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.39/containers/create?name=gridstudio: open //./pipe/docker_engine: The system cannot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to connect. This error may also indicate that the docker daemon is not running.
See 'C:\Program Files\Docker\Docker\Resources\bin\docker.exe run --help'.

Case sensitivity for cell names and function names

Case sensitivity, both in cell names ("A1" vs "a1") and function names. For one who has spent any amount of time using Excel or LibreOffice, this is going to be a huge hurdle.

Contributed by: rcfox

pd.read_csv doesn't work

Hi,rick.I fail to use pd.read_csv to load a file in gridstudio though I can load the same file in spyder with same path. Here is error log
image.

Issue cloning repository

The files under the 'gridstudio/grid-app/static/xterm/node_modules/.cache/uglifyjs-webpack-plugin/content-v2/sha512/' have filenames that are too long on Windows 10. The repository can't be cloned completely.

Screenshot below:
https://imgur.com/a/E4pSYGN

Troubleshooting(Help Needed): Not responding?

I am using Ubuntu 18.04LTS. Docker is already configured and updated to the newest version.
Using AWS EC2 instance.
I am encountering the problem that the process never stops... I tried to run HelloWorld but it keeps on computing for more than 10 mins and it's still computing. Need some help!
Uploading snip.png…
Uploading snip2.png…

Problem installing on Windows Server 2019

Hi,

I brought the install to the point of running run.sh on a Git bash console. Then I get:

C:\Program Files\Docker\docker.exe: image operating system "linux" cannot be used on this platform.

So the image is a linux image and cannot be run in Windows? How to sort this out?

Thanks for helping.

Can the storage space save the picture as well?

I output the data to the excel table and find that the result is acceptable.This is how I operate:
sql="select ID,type_NO from f_personal_info where rownum<5"
data = pd.read_sql(sql,connection)
print(data)
data.head(n=2)
sheet("A",data,headers=True)

image

However, when the workspace I saved was reopened, the image data did not exist.

How can this be solved?

Thank you very much

Not working on Win7 running Docker Toolbox

I successfully downloaded the docker image and it shows in docker images. But I get the following error after ./run.sh on Git Bash for Win7 running Docker Toolbox:

$ ./run.sh
Error response from daemon: No such container: gridstudio
Error: failed to start containers: gridstudio

I can run the container using docker run ricklamers/gridstudio:release which gives the following output (same as docker logs <CONTAINER>:
Screenshot_1

Since I'm running Docker Toolbox, my default IP is 192.168.99.100, but I get a "Unable to connect" error when visiting http://192.168.99.100:8080.

And here's the output from docker container ls:
Screenshot_2

Also, the folder is located in C:\Users\sina\repos\gridstudio.

create workspace error

I use the docker-compose file to run the server on Mac OS

version: "2"
services:
  gridstudio:
    image: ricklamers/gridstudio:release
    volumes:
      - /labs/gridstudio/grid-app:/home/source
      - /labs/gridstudio/grid-app/proxy/userdata:/home/userdata
    ports:
      - 38081:8080

Login succeeds, but an error pops up when trying to enter or create a workspace.

1

Error logs from docker

ridstudio_1  | [Spawn] Tried creating processes GoCmd and NodeCmd
gridstudio_1  | 2019/07/31 12:46:50 http: panic serving 172.23.0.1:42072: runtime error: invalid memory address or nil pointer dereference
gridstudio_1  | goroutine 65 [running]:
gridstudio_1  | net/http.(*conn).serve.func1(0xc00007f220)
gridstudio_1  |         /usr/local/go/src/net/http/server.go:1769 +0x139
gridstudio_1  | panic(0x890ee0, 0xe35fa0)
gridstudio_1  |         /usr/local/go/src/runtime/panic.go:522 +0x1b5
gridstudio_1  | main.destructSession(0xc0001186ce, 0x24, 0xc000072e10, 0xc000072e40)
gridstudio_1  |         /home/source/proxy/manager.go:217 +0x128
gridstudio_1  | main.main.func16(0x98bc60, 0xc000146a80, 0xc00027e900)
gridstudio_1  |         /home/source/proxy/manager.go:778 +0xfb
gridstudio_1  | net/http.HandlerFunc.ServeHTTP(0xc00007c460, 0x98bc60, 0xc000146a80, 0xc00027e900)
gridstudio_1  |         /usr/local/go/src/net/http/server.go:1995 +0x44
gridstudio_1  | net/http.(*ServeMux).ServeHTTP(0xe71d60, 0x98bc60, 0xc000146a80, 0xc00027e900)
gridstudio_1  |         /usr/local/go/src/net/http/server.go:2375 +0x1d6
gridstudio_1  | net/http.serverHandler.ServeHTTP(0xc000078d00, 0x98bc60, 0xc000146a80, 0xc00027e900)
gridstudio_1  |         /usr/local/go/src/net/http/server.go:2774 +0xa8
gridstudio_1  | net/http.(*conn).serve(0xc00007f220, 0x98c460, 0xc000054f80)
gridstudio_1  |         /usr/local/go/src/net/http/server.go:1878 +0x851
gridstudio_1  | created by net/http.(*Server).Serve
gridstudio_1  |         /usr/local/go/src/net/http/server.go:2884 +0x2f4
gridstudio_1  | Tried destroying session 3091f371-61ad-44bc-9242-1fce927ca53f, but sessions not in active usersessions.
gridstudio_1  | 0 user sessions active.

miss file 'main.py'

Error calling getFile for path: /home/userdata/workspace-a39ce24b-83ec-43c7-8219-522d3aaf4eef/userfolder/main.py
open /home/userdata/workspace-a39ce24b-83ec-43c7-8219-522d3aaf4eef/userfolder/main.py: no such file or directoryReceived WS in Client actions: {"arguments":["SWITCHSHEET","0"]}

Click error

When I selected one cell, if I click another cell, the cell selected previous will get the wrong content.

Doesn't install/run

Install says:

  1. Run the bash script (on Windows use e.g. Git Bash) with this command:

cd gridstudio && ./run.sh

fish shell says:

/run.sh
Failed to execute process './run.sh'. Reason:
exec: Exec format error
The file './run.sh' is marked as an executable but could not be run by the operating system.

bash says:

./run.sh
./run.sh: line 12: docker: command not found

I'm on ubuntu19.04

releases

start doing releases would be great. possibly versions for docker as well pulling a git release

WSL cannot connect to the Docker daemon

Installed on latest Windows Subsystem Linux (Ubuntu kernel). Unable to run run.sh, failed with error

docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.

Steps to reproduce:

  1. start WSL
  2. apt-get install docker.io
  3. git clone repo and cd into gridstudio-master
  4. run ./run.sh

Fixes attempted:

  1. sudo (naturally)
  2. apt-get update
  3. checked docker run --help to ensure docker was installed and running

line 5: docker: command not found

when run "./run.sh", it gives me this:

$ cd gridstudio && ./run.sh
./run.sh: line 5: docker: command not found
./run.sh: line 11: docker: command not found

and I went to http://127.0.0.1:8080, I got can't reach this page, how can I fix the issue? Thank you!

Run run-manager-proxy.sh: Permission denied

Hi dear,

New for docker and grid studio, The error msg shows as below after running ./run.sh:
bash: run-manager-proxy.sh: Permission denied

Please help to give me some advice! thx!

What contribution can I make?

I am a student in Xi'an, China. When I saw your open source project, I studied and installed it in the first time. It was very interesting, but I don't know if I can help you. I studied statistics, and the tool I was good at was R language.

If you have any place where you need me, please feel free to tell me. I have plenty of time. I hope this tool can be used more in China.

Big parser rewrite

I'd like to give everyone a heads up that I'm rewriting the function parser in: https://github.com/alecthomas/participle

It's progressing nicely - but for all contributers please keep in mind that this will have a significant impact across the codebase of the Go backend. It will now work on the parsed AST instead of the raw string functions which will impact pretty much all Go back-end code.

after login it says 'attempt to write a readonly database'

$ docker run --name=gridstudio --rm=false -v E:/gridstudio/grid-app:/home/source -v E:/gridstudio/grid-app/proxy/userdata:/home/userdata -p 8080:8080 -p 4430:4430 ricklamers/gridstudio:release
--> Updating terminal app.js
--> Run manager proxy, starting manager.go (compiling with go run ...)
Listening on port: 8080
WS Listening on port: 4430
Hashed PW: 19yxYLFN-1h1mow8cpqDqLlgsHo=
2019/08/02 03:21:26 attempt to write a readonly database
Login, found user matching hashed PW.
exit status 1

Issue with path conversion in the run.sh with Docker Desktop for Windows

The run.sh script fails while using Git Bash on Windows

 MINGW64 /d/dev/gridstudio (master)
$ ./run.sh
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: Mount denied:
The source path "D:/dev/gridstudio/grid-app/proxy/userdata;D"
doesn't exist and is not known to Docker.
See 'C:\Program Files\Docker\Docker\Resources\bin\docker.exe run --help'.

Related: https://stackoverflow.com/questions/50608301/docker-mounted-volume-adds-c-to-end-of-windows-path-when-translating-from-linux

Changing it to this fixed it:
docker run --name=gridstudio --rm=false -v /${PWD}/grid-app:/home/source -v /${PWD}/grid-app/proxy/userdata:/home/userdata -p 8080:8080 -p 4430:4430 ricklamers/gridstudio:release

Can not execute the example/estimate_normal.py

step1. start the grid studio and log in the website
step2. copy the example/estimate_normal.py to the script window
step3. execute
However, the error msg showed like,
Traceback (most recent call last):
File "python/init.py", line 282, in getAndExecuteInput
File "", line 1, in
NameError: name 'time' is not defined

Expand autofill UX to draggable

Currently only works by copy pasting

Needs an auto-fill feature. The first thing I do in pretty much any spreadsheet is set A1 to 1 and A2 to =A1+1, select bunch of cells below and auto-fill down.

Contributed by: rcfox

how to exit?

looks like ctrl+c does not work to stop running service in console

Lost connection to the server. Redirect to dashboard?

centos7,docker18.09.7

1、installed the grid studio;success

2、user the root account run the cmd:./run.sh

[root@publicmariadb gridstudio]# ./run.sh
--> Updating terminal app.js
--> Run manager proxy, starting manager.go (compiling with go run ...)
Listening on port: 8080
WS Listening on port: 4430
Hashed PW: 19yxYLFN-1h1mow8cpqDqLlgsHo=

3、open chrom http://127.0.0.1:8080
**4、login:admin/admin
5、for 5-10 seconds,pop the eror "Lost connection to the server. Redirect to dashboard?"**Lost connection to the server. Redirect to dashboard?

can‘t start

bash: run-manager-proxy.sh: No such file or directory

how to fix it ,please

How is the limit of opening CSV file?

My CSV file contains about 700,000 rows x 61 columns (57 MB). And it cannot be opened in gridstudio, even my computer (8GB RAM running Ubuntu 18.04) stopped response. Can this software open large file? But the file can be directly imported using Pandas in Python.

Docker container does not start when executing run.sh

There seems to be an issue with the Dockerfile.

$ docker logs gridstudio
bash: run-manager-proxy.sh: No such file or directory
bash: run-manager-proxy.sh: No such file or directory
bash: run-manager-proxy.sh: No such file or directory

127.0.0.1 refused to connect.

I followed the installation steps and ran in to the same problem as Soarnsky. I used the code he posted to replace the 'cd gridstudio && ./run.sh and then I was able to get past that error. However, when I go to 127.0.0.1:8008 I get the follow error in Chrome: This site can’t be reached 127.0.0.1 refused to connect. I've confirmed Hyper-V is on and Visualization is active in the BIOS. Any ideas on how to fix this?

Issue running on Mac OS X Mojave

run.sh on my Mac yields the following error message (note, I'm running via sudo since /var/run apparently needs privilege to write the unix domain socket file.)

:; sudo ./run.sh
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.

--

Note that there is no such file:

:; ls -al /var/run/dock*
ls: /var/run/dock*: No such file or directory

--

I also killed a docker-related helper daemon prior to this; there are no docker-related processes running when I run the runscript.

ps ax | grep -i docker
22686 s003 S+ 0:00.00 grep -i docker

Thoughts?

jb

Could pls give a step by step guide?

I am using Pycharm and just cloned this project. But I can't find a way to start using it.

Which scripts should I run or command to input in where to get the grid to pop out like using Jupter (aka: command> Juputer notebook > then the page came out)?

Sorry for this stupid question since I just know this tool via a article. Thanks!

Clarify and maybe enhance reproducibility / transparency

Setting up an issue based on our brief exchange on Reddit.

It would probably go a long way simply to point people to the text file record of commands.

I'd talked about a Gigantum integration (which could happen - but perhaps isn't a priority given your current cloud push, and I don't want to undercut that). But some ideas you could carry over include keeping thumbnails and outputs in addition to commands.

File-Load CSV doesn't work

I have install this tool successfully, but when I click the button "File-Load CSV", Nothing happen. The button only worked is Help-Documentation&Slack community. What's wrong with me?

run-manager-proxy.sh missing after docker run command

I simply followed the rules on https://github.com/ricklamers/gridstudio/wiki/Installation and then it failed:

XXXX@YYYY MINGW64 /c/MaLiTuP_Data/gridstudio (master)
$ sh run.sh
Unable to find image 'ricklamers/gridstudio:release' locally
release: Pulling from ricklamers/gridstudio
7413c47ba209: Pulling fs layer
0fe7e7cbb2e8: Pulling fs layer
1d425c982345: Pulling fs layer
344da5c95cec: Pulling fs layer
ad781393c0f2: Pulling fs layer
247beeb00eac: Pulling fs layer
64ac51b6ebe0: Pulling fs layer
63856feadb4f: Pulling fs layer
0b79ec7460fe: Pulling fs layer
23fb864bc232: Pulling fs layer
f8a98446c6c2: Pulling fs layer
36f4f2833339: Pulling fs layer
a84809321185: Pulling fs layer
7f590f724642: Pulling fs layer
a3101cd01c82: Pulling fs layer
8a45d0a0c102: Pulling fs layer
124542e12ed2: Pulling fs layer
7b2c0a114508: Pulling fs layer
7eab9d775b7c: Pulling fs layer
344da5c95cec: Waiting
ad781393c0f2: Waiting
247beeb00eac: Waiting
64ac51b6ebe0: Waiting
63856feadb4f: Waiting
0b79ec7460fe: Waiting
23fb864bc232: Waiting
f8a98446c6c2: Waiting
36f4f2833339: Waiting
a84809321185: Waiting
7f590f724642: Waiting
a3101cd01c82: Waiting
8a45d0a0c102: Waiting
124542e12ed2: Waiting
7b2c0a114508: Waiting
7eab9d775b7c: Waiting
0fe7e7cbb2e8: Verifying Checksum
0fe7e7cbb2e8: Download complete
1d425c982345: Verifying Checksum
1d425c982345: Download complete
7413c47ba209: Download complete
344da5c95cec: Download complete
ad781393c0f2: Verifying Checksum
ad781393c0f2: Download complete
7413c47ba209: Pull complete
0fe7e7cbb2e8: Pull complete
63856feadb4f: Verifying Checksum
63856feadb4f: Download complete
1d425c982345: Pull complete
344da5c95cec: Pull complete
ad781393c0f2: Pull complete
0b79ec7460fe: Verifying Checksum
0b79ec7460fe: Download complete
23fb864bc232: Verifying Checksum
23fb864bc232: Download complete
f8a98446c6c2: Verifying Checksum
f8a98446c6c2: Download complete
247beeb00eac: Verifying Checksum
247beeb00eac: Download complete
a84809321185: Verifying Checksum
a84809321185: Download complete
7f590f724642: Verifying Checksum
7f590f724642: Download complete
64ac51b6ebe0: Verifying Checksum
64ac51b6ebe0: Download complete
8a45d0a0c102: Verifying Checksum
8a45d0a0c102: Download complete
36f4f2833339: Verifying Checksum
36f4f2833339: Download complete
a3101cd01c82: Verifying Checksum
a3101cd01c82: Download complete
7eab9d775b7c: Verifying Checksum
7eab9d775b7c: Download complete
7b2c0a114508: Verifying Checksum
7b2c0a114508: Download complete
124542e12ed2: Verifying Checksum
124542e12ed2: Download complete
247beeb00eac: Pull complete
64ac51b6ebe0: Pull complete
63856feadb4f: Pull complete
0b79ec7460fe: Pull complete
23fb864bc232: Pull complete
f8a98446c6c2: Pull complete
36f4f2833339: Pull complete
a84809321185: Pull complete
7f590f724642: Pull complete
a3101cd01c82: Pull complete
8a45d0a0c102: Pull complete
124542e12ed2: Pull complete
7b2c0a114508: Pull complete
7eab9d775b7c: Pull complete
Digest: sha256:c7b2d9aa20ea9903d2ebce119a094114d829f54423c5ab59e0ec23b8ee819905
Status: Downloaded newer image for ricklamers/gridstudio:release
bash: run-manager-proxy.sh: No such file or directory

As I was only pulling the image from remote (no local changes could have affected anything), I guess the current docker image on docker-hub is broken?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.