Comments (16)
#283 was not the right approach. we have yet to find a solution to this.
from superset.
that would really help! I knew this was going to hit on some db with different autocast...
from superset.
HI,
Haven't found a nice way to let SQLAlchemy dialect to convert the date time yet.
BTW I'm managing with this patch/workaround at models.py line 542*
# UGLY: I guess correct way is to delegate on SQLAlchemy dialect
# UPDATE: Datetime depends on each dialect and I haven't found an easy way to manage
# Maybe we can allow user to define its custome format at Database definition
def get_dtformat(type):
if type == 'SMALLDATETIME' or type == 'DATETIME':
return '%Y-%m-%d %H:%M:%S'
if type == 'DATE':
return '%Y-%m-%d'
if type == 'TIME':
return '%H:%M:%S'
return '%Y-%m-%d %H:%M:%S.%f'
tf = get_dtformat(cols[granularity].type or 'DATE')
*Sorry for posting code, I'm facing some issues managing new branches and PR.
from superset.
Thanks for the workaround .
from superset.
I am still finding problem with this issue. I just updated to latest caravel.
Should I do anything to make this #283 fix to work?
from superset.
This should help #446, please test and report whether it fixes your issues.
from superset.
@gbrian is this fixed for you?
from superset.
Hi @xrmx,
Sorry not working due to precision:
Don't know the best fix, I'll suggest trim precision.
http://stackoverflow.com/questions/19025192/convert-varchar-to-datetime-in-sql-which-is-having-millisec/38843397#38843397
from superset.
@xrmx : If you agree we can use column data type to decide if we can use DateTime2 or trim milliseconds. I mean in case column has been defined as DateTime2 cast to DateTime2, if not just trim (but not TSQL version).
Will this work for you?
from superset.
@gbrian adding a proper python_date_format wouldn't fix that? I know it's manual and not working out of the box is lame.
from superset.
@xrmx proper way (IMO)
-- Using default and 126 format for Datetime2
SELECT CONVERT(DATETIME2, '2015-08-10 11:58:47.123456', 126) as MSSQLDateTime2
-- Using default[:-3] and 121 format for Datetime
SELECT CONVERT(DATETIME, '2015-08-10 11:58:47.123', 121) as MSSQLDateTime2
so
some changes in the code like:
def dttm_sql_literal(self, dttm, type):
"""Convert datetime object to string
If database_expression is empty, the internal dttm
will be parsed as the string with the pattern that
the user inputted (python_date_format)
If database_expression is not empty, the internal dttm
will be parsed as the sql sentence for the database to convert
"""
tf = self.python_date_format or '%Y-%m-%d %H:%M:%S.%f'
if self.database_expression:
return self.database_expression.format(dttm.strftime('%Y-%m-%d %H:%M:%S'))
elif tf == 'epoch_s':
return str((dttm - datetime(1970, 1, 1)).total_seconds())
elif tf == 'epoch_ms':
return str((dttm - datetime(1970, 1, 1)).total_seconds() * 1000.0)
else:
default = "'{}'".format(dttm.strftime(tf))
iso = dttm.isoformat()
d = {
'mssql': "CONVERT({}, '{}', {})".
format("DATETIME2" if type.lower() == "datetime2" else "DATETIME",
default if type.lower() == "datetime2" else default[:- 3],
126 if type.lower() == "datetime2" else 121), # untested
'mysql': default,
'oracle':
"""TO_TIMESTAMP('{}', 'YYYY-MM-DD"T"HH24:MI:SS.ff6')""".format(iso),
'presto': default,
'sqlite': default,
}
for k, v in d.items():
if self.table.database.sqlalchemy_uri.startswith(k):
return v
return default
I think is this way I keep as much precision as possible with the definition the user has done.
from superset.
@gbrian a diff is easier to review :) Anyway the chain of ifs insinde the format is a no-go imho :) Also we can remove the for loop and the dict if we need to patch only mssql and oracle. Also in mssql case do we really need the CONVERT, can't we just return a slice of default instead?
uri = self.table.database.sqlachemy_uri
if uri.startswith('oracle'):
iso = dttm.isoformat()
return """TO_TIMESTAMP('{}', 'YYYY-MM-DD"T"HH24:MI:SS.ff6')""".format(iso)
elif uri.startswith('mssql'):
field_type = type.upper()
if field_type == 'DATETIME':
return default[:-3]
return default
else:
return default
from superset.
Yeah! totally agree with the "ifs" ;) was just playing around, sorry. Sadly I don't have an older SQL Server version to test the "default" behavior so I though CONVERT will be more backward compatible: http://www.techonthenet.com/sql_server/functions/convert.php (basically will cover SQL Server 2005)
Let me try again:
from superset.
@gbrian do we really need the convert for the DATETIME2? it looks like it can handle our default just fine.
Anyway open a pull request, looks pretty good anyway and an improvement over current code :)
BTW please reorder the ifs to check for mssql first, i moved there just for c&p convenience :)
from superset.
Ah, ok! yep you are right, for DATETIME2, CONVERT is not needed. Creating PR, talk later
from superset.
What is the point of formatting dates within Superset, why don't let SQLAlchemy do that?
from superset.
Related Issues (20)
- Superset HOT 1
- Unable to execute 'ecpg' , you likely need to install it
- Extracted Translation Order HOT 1
- BUG: Can't flush None value found in collection Role.permissions
- sqlalchemy engine should be created once per process
- Cannot use drill-by/drill-to without can explore on Superset permission
- Not able to connect MySQL
- superset init fail if not run just after superset db upgrade HOT 1
- Unable to load SuperSet UI when I'm using via a domain HOT 7
- Latest helm chart points to 3.1.1 instead of 3.1.2 HOT 1
- empty result dashboard from superset APIs `/api/v1/dashboard` when the Public role has `can read on Dashboard` and `can read on Chart` . HOT 1
- Wrong url while getting userinfo from keycloak HOT 1
- Facing issues with connecting to SQL lab after updating to superset 3.1.2 HOT 5
- huge memory consumption while running frontend unit testcases locally
- Enabling partial text matches in dashboard filters
- Helm chart installation times out and starts to consume huge amount of CPU HOT 5
- Metric Key instead of Label in Line Chart plot and legend
- Report quality is not good in mobile slack app. HOT 1
- Problem when creating dataset/chart/dashboard if database is Trino with TRINO_EXPAND_ROWS
- Drill by is disabled for Timeseries and Mixed Timeseries charts without dimensions HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from superset.