Cannot create tables out of SQL Select statements about dbt-synapse HOT 14 CLOSED

Hi alieus and swanderz,

i figured out the solution for it. i need to change my data type to make it work. Using convert operator in the sql syntax worked.

Thank you for your kind help. you may close the ticket.

from dbt-synapse.

what type of Azure synapse dedicated pools are you using: dedicated or serverless? My guess is you're using a serverless pool which do not support tables. You may want to try dbt-msft/dbt-synapse-serverless instead. Though huge caveat that serverless pools aren't that great of a solution for synapse today.

from dbt-synapse.

Hi swanderz , could not reply you immediately.
I am using a dedicated SQL pool. No luck with table creation.

from dbt-synapse.

weird! here's some questions:

1. what version of dbt-synapse are you using?
2. are you using seeds or sources for these transformations?
3. what is the column type of device_id?
4. can you share the info from the latest run in the log (logs/dbt.log)?

from dbt-synapse.

1. • sqlserver: 0.20.1
   • synapse: 0.20.0

2. Sources yes
   

4. dbt.log

from dbt-synapse.

Thanks @Rahul-Gajula. I’ll try to repro this error and see what I find

from dbt-synapse.

Looking at the error message I see:

Database error: ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]The statement failed. Column 'device_id' has a data type that cannot participate in a columnstore index.

Any chance you can change your column types to not use nvarchar(max)? Not only will it not work with dbt-synapse, but it's not a good idea as it isn't very optimized.

from dbt-synapse.

@swanderz you're absolutely right. Running this directly in Synapse validates that you cannot have nvarchar(max) as a datatype:

However, the below works with no errors:

from dbt-synapse.

Hi alieus and swanderz,

Tried creating table as per the comments. Failed yet again.

dbt.log

It's not even recognizing the Create Syntax

from dbt-synapse.

Hey @Rahul-Gajula , if you’d like to test this, you’d have to run it directly in your synapse workspace.

What you need to do is change the datatypes in your source tables from nvarchar(max) to nvarchar(n) and then run your select queries again in dbt.

from dbt-synapse.

Hi alieus,

Even with the data type nvarchar(n) i am not able to create a table.

CREATE TABLE telemetry_test(
device_id NVARCHAR(n),
IP_adress NVARCHAR(n)
);

from dbt-synapse.

where n is the size of the string in byte-pairs. For example, nvarchar(16).

from dbt-synapse.

Hi alieus,

Even that i tried. refer to the screen shot.

from dbt-synapse.

Glad to hear it! I’m sure this issue will definitely help someone else in the future.

from dbt-synapse.