Comments (4)
Testing with the pre-release this looks like it's working properly to me!
from sling-cli.
Not sure if this is feasible... What types of non-utf8 encodings are you thinking of?
from sling-cli.
latin-1
is the one I encounter most but I understand if this isn't worth pursuing or isn't possible. At a certain point you can just say data in the databases should be utf-8
and there should be processes to make sure weird characters don't get in there
from sling-cli.
Hey, I pre-released a new version: https://github.com/slingdata-io/sling-cli/releases/tag/v1.0.31
Can you download the binary, test and let me know if it works? You can use the transforms: [ decode_latin1 ]
option like this:
source: SNOWFLAKE_DEV
target: DOCKER_POSTGRES
defaults:
mode: truncate
source_options:
transforms: [ decode_latin1 ]
streams:
snowflake_schema.snowflake_table:
object: public.my_postgres_table
Here are the options: https://github.com/slingdata-io/sling-cli/blob/main/core/sling/tranforms.go#L20-L27
from sling-cli.
Related Issues (20)
- mssql: The ORDER BY clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions, unless TOP, OFFSET or FOR XML is also specified. HOT 3
- Don't Create Parquet File with 0 rows HOT 1
- Sling crashes when migrating table from postgres to s3 HOT 6
- Improve Logic for `NULL` vs blank values from source (database -> database) HOT 1
- MSSQL Date checksum failing due to overflow HOT 2
- `_sling_loaded_at` added to table when loading files from s3/gcs (mode: full-refresh) HOT 3
- MongoDB - Unable to connect to source HOT 5
- BCP over SSH HOT 1
- MongoDB - Empty ISODate column in Postgres destination HOT 2
- File destination - have a skip option for same file name instead of replace/delete HOT 8
- Incremental mode not working for file to database sync HOT 4
- Oracle DATE column should create DATETIME in target db HOT 2
- MySQL to Snowflake - empty date field give an error HOT 4
- Skip column during replication HOT 1
- OpenID Authentication to BigQuery on GitHub Action HOT 7
- SFTP written as STFP in docs HOT 1
- Default incremental mode prevents using full-refresh on stream level HOT 4
- Add DefaultAzureCredential as an authentication option for Azure HOT 1
- Sling padding varchar columns to max length (SQL Server -> Snowflake)
- Panic while exporting Trino table to local parquet file HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sling-cli.