Row Level security allows DataHub to maintain a table detailing what users have Select/Insert/Alter privileges on other tables.

This is confusing to the user. The "reset" link should be present on the login page, initially.

I'm copying a few of the frequently requested features/fixes from jira onto Github, just so people know that we're aware of the issues, and are working on them.

They're currently visible at https://github.com/abhardwaj/datahub/blob/master/_build/html/index.html

This seems to be a missing (non-vital) formatting package. Unfortunately, I'm unfamiliar with both Go and Thrift.

https://golang.org/cmd/gofmt/

Here's the error:

$ cd src/examples/go
$ /.setup.sh

sh: gofmt: command not found WARNING - Running 'gofmt -w /Users/arcarter/code/datahub/src/examples/go/gen-go/src///datahub/datahub.go' failed. sh: gofmt: command not found WARNING - Running 'gofmt -w /Users/arcarter/code/datahub/src/examples/go/gen-go/src///datahub/ttypes.go' failed. sh: gofmt: command not found WARNING - Running 'gofmt -w /Users/arcarter/code/datahub/src/examples/go/gen-go/src///datahub/constants.go' failed. sh: gofmt: command not found WARNING - Running 'gofmt -w /Users/arcarter/code/datahub/src/examples/go/gen-go/src///datahub/account/account_service-remote/account_service-remote.go' failed. sh: gofmt: command not found WARNING - Running 'gofmt -w /Users/arcarter/code/datahub/src/examples/go/gen-go/src///datahub/account/accountservice.go' failed. sh: gofmt: command not found WARNING - Running 'gofmt -w /Users/arcarter/code/datahub/src/examples/go/gen-go/src///datahub/account/ttypes.go' failed. sh: gofmt: command not found WARNING - Running 'gofmt -w /Users/arcarter/code/datahub/src/examples/go/gen-go/src///datahub/account/constants.go' failed.

1.5 is no longer supported.

I'm linking github and Jira issues. This is a test to see if Jira picks up new github issues. Apologies for the notifications.

the current layout.html is not a compiled template (it's a raw template) -- in future we would compile it so that it separates out the static files (that can be server through either CDN) or a fast web server like nginx and put the template handling inside a reverse proxy. I'll put up a task for it but it's not super important.

Anant

In the datahub's layout.html we're importing 30+ header files. Many of these are for specific applications (i.e. terminal), and are hosted locally instead of using a CDN. Is this really necessary?

My impression is that it'd be faster/cleaner to put these in the application headers, and load them from cdn networks where possible.

Albert Carter

`connect to livinglab; != connect to livinglab``

Datahub interprets the semicolon as part of the username.

This totally doesn't exist.

The following piece of code throws the above error:

this.transport = new THttpClient("http://datahub.csail.mit.edu/service");
this.protocol = new  TBinaryProtocol(transport);
this.client = new DataHub.Client(protocol);

this.con_params = new ConnectionParams();
this.con_params.setUser("anantb");
this.con_params.setPassword("anant");
this.conn = this.client.open_connection(con_params);

ResultSet updatelogExists =  this.client.execute_sql(this.conn, "select * from anantb.test.demo", null);

Gives the following error:

DBException(message:User matching query does not exist. Lookup parameters were {'username': None})...

Would appreciate a fix for this! Thanks!

{"error": "\"viewtest\" is not a table\nHINT: Use DROP VIEW to remove a view.\n"}

table_delete in browser.views will need to determine if the table_name passed is a view, and it will have to call related methods in manager.py and pg.py

1. access control. is there any place I can look to see all the tables "shared with me" (ie to which I have access privileges)?

@karger

DataTables warning: table id=table_data - Ajax error. For more information about this error, please see http://datatables.net/tn/7

Happens when I load most tables

No matter what I enter in, I am unable to create an account (and thus to do anything with datahub at all).

The "Apps Center" link on the top nav bar currently points to root. There should either be a list of installed apps, or the link should be removed.

core.db.backends.pg.PGBackend.get_schema already even exists, and is used by apps.

Allow users to share query URLs with each other. Related to #82

"core" just looks like a bad idea.

its not possible for users to delete repositories.

https://datahub.csail.mit.edu/browse/username

click trash can icon

Load Modal

Click Delete

The modal does not disappear. There is no confirmation dialog. Nothing happens.

reported by @karger:

In javascript I'm using the thrift api to send datahub a sql query that
includes the "array_agg" operator:
"select count(prequest.hilulim.uid) as num, prequest.hilulim.title as
title ,array_agg(prequest.names.name) as names from prequest.hilulim
join ...."

the aggregated column is being returned by the thrift api as a string
encoding of an array of unicode strings which is really weird---ie, the
value in the cell is the string "[u'Fiana Sara Eber', u'David Karger']"
Why am I getting this python-language syntax exposed in theoretically
language-agnostic thrift? Why isn't coming back as an array of
strings? why the unicode encoding? will it always be a unicode
encoding? do I need to parse it myself? Unfortunately since it's a
python encoded string I can't hand it to JSON.parse()

If the username provided is lowercase, the thrift files throw an unknown exception

Users might be able to add javascript applications that affect tables through the DataHub API, for example Hands On Table
@karger

Go to the console, then ls reponame

Base tables will show, but views won't.

This happens because the list_tables method now only lists base tables. Unfortunately, I'm having trouble getting Thrift (0.9.2) to generate up-to-date javascript code which will allow a javascript client.ls_views function.

It should be possible disconnect from the current repo in the terminal. In postgres, this should be with the disconnect command and/or ctrl+d, but I'm not sure how other databases manage it.

account email addresses are case sensitive. It's possible to create an account with [email protected], and then another with [email protected], and then give them separate usernames and passwords.

in objective-c:

[account_client create_account:username email:@"[email protected]" password:password repo_name:@"getfit" app_id:appID app_token:appToken];

and then create another account:

[account_client create_account:username email:@"[email protected]" password:password repo_name:@"getfit" app_id:appID app_token:appToken];

When users don't specify a schema for table creation, the table ends up in PUBLIC. The system should warn them of this, and possibly disallow tables in the public schema.

Some tables should be public, so that all users can create views off of them. This ties into another problem, discoverability.

The table owner sees pagination, but DataTables does not show pagination for the non-owner

@karger

/home/ubuntu/datahub/src/apps/dbwipes/views.py:6: DeprecationWarning: the md5 module is deprecated; use hashlib instead
import md5

Internal Server Error: /
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 103, in get_response
resolver_match = resolver.resolve(request.path_info)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 319, in resolve
for pattern in self.url_patterns:
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 347, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 342, in urlconf_module
self._urlconf_module = import_module(self.urlconf_name)
File "/usr/local/lib/python2.7/dist-packages/django/utils/importlib.py", line 35, in import_module
import(name)
File "/home/ubuntu/datahub/src/browser/urls.py", line 171, in
url(r'^apps/dbwipes/', include('dbwipes.urls')), # dbwipes app
File "/usr/local/lib/python2.7/dist-packages/django/conf/urls/init.py", line 25, in include
urlconf_module = import_module(urlconf_module)
File "/usr/local/lib/python2.7/dist-packages/django/utils/importlib.py", line 35, in import_module
import(name)
File "/home/ubuntu/datahub/src/apps/dbwipes/urls.py", line 2, in
import views
File "/home/ubuntu/datahub/src/apps/dbwipes/views.py", line 16, in
from service.handler import DataHubHandler
File "/home/ubuntu/datahub/src/service/handler.py", line 7, in
from datahub import DataHub
ImportError: No module named datahub
[27/Apr/2015 03:19:12] "GET / HTTP/1.1" 500 122244

It's possible to GRANT VIEW to schemas, but not through the interface. This should be possible.

I would like to be able to link people to tables/repositories without using their usernames:

for example, instead of
https://datahub.csail.mit.edu/browse/USERNAME/REPONAME/table/TABLENAME

use this and have username inferred by their login status.
https://datahub.csail.mit.edu/browse/REPONAME/table/TABLENAME

This is a thing that kept bugging me during getfit.

DataHub adds automatic limit statements to all sql statements in repository interface (http://datahub.csail.mit.edu/browse/USERNAME/REPONAME/), to support pagination. As a result, a statement like select * from getfit.deviceinfo LIMIT 1; should work, but instead returns an error:

{"error": "syntax error at or near \"LIMIT\"\nLINE 1: select * from getfit.deviceinfo LIMIT 1 LIMIT 50 OFFSET 0\n ^\n"}

When a user creates a card using DML, they are asked to give it a name. Currently, names that contain blank spaces cannot be mapped to urls, and break.

When using the console, you sometimes have to click more than once and try to paste more than once before any text appears.

I'm using OSX and chrome.

This is possibly one of the simplest and most frustrating parts of datahub... because I no one wants to be editing sql in a terminal.

Maybe save create view statements as cards, or save all queries as temporary cards? It's too hard to retrieve queries that have previously been run successfully.

http://datahub.csail.mit.edu/

At present, it's just redirecting to home. Last week, it was 404-ing

Seriously, add a requirements.txt or setup.py or just a list to the README.

It takes a large number of lines to create a client. Maybe this could be done with one line and assume a default http client. If the user wanted a TCP client, they could specify.

Doing big queries over HTTP is incredibly time consuming. Gzipping is pretty straightforward, and would significantly speed up load time.

http://datahub.csail.mit.edu/browse/al_carter/test/tables

click the '+' button

http://datahub.csail.mit.edu/browse/al_carter/test/files

Manipulating large datasets in the terminal causes it to hang. The only way to interrupt is to force quit the tab. It should be possible to keep listening for ctrl+c and interrupt the process if the user desires.

Currently, the DataQ app is not accessible via the DataHub user interface. Add a button to table-browse-template.html to launch the app.

datahub.csail.mit.edu <-- works
www.datahub.csail.mit.edu <-- doesn't work