Default to `utf-8` encoding about tap-csv HOT 3 CLOSED

Agreed with making utf-8 the default. I don't think there's a strong need to require system-specific encoding (these things should be portable across OSes), but @edgarrmondragon's suggestion of passing an explicit null value would make sense to me. (More likely though we'd recommend them to send latin1 or another explicit encoding name.)

from tap-csv.

@visch thanks for opening this!

TIL that open() uses your machine's preferred encoding. I was expecting it to be something consistent under the hood but I see why this can be confusing because I think running the tap on the same file with same configs on two different machines could produce different outputs.

encoding is the name of the encoding used to decode or encode the file. This should only be used in text mode. The default encoding is platform dependent (whatever locale.getpreferredencoding() returns), but any text encoding supported by Python can be used. See the codecs module for the list of supported encodings.

After learning that I'd agree that choosing a default for ourselves in the tap is a good idea. But I'm not all that versed in this stuff, especially cross platform compatibility, so I'd like to hear others opinions.

from tap-csv.

UTF-8 would be a sane default. Users could still let Python use the system's encoding by configuring "encoding": null for each file.

from tap-csv.