UTF-8 support about nubasic HOT 6 OPEN

from nubasic.

I would prefer Latin-1 ( = ISO 8859-1 ), or a switch between ISO-8859-1 and UTF-8.
The reason is: My editors assume that .bas-Files are encoded in ISO 8859-1.

from nubasic.

I would prefer Latin-1 ( = ISO 8859-1 ), or a switch between ISO-8859-1 and UTF-8.

I'm not sure what you mean by "switch".

The reason is: My editors assume that .bas-Files are encoded in ISO 8859-1.

That is a problem of your editors ;)

Unicode is the way to go, and UTF-8 is its most practical encoding at the moment. Of course, it brings the issue about the BASIC string functions, but they could work with bytes as usual. The thing is to accept and print UTF-8 strings.

But anyway ISO 8859-1 is better than nothing: it would make nuBASIC useful to write programs in a few European languages other than English.

from nubasic.

Such limitation was simplifying the implementation, but maybe now I can improve it.

Thanks. I understand ASCII was enough for your initial scope, but it makes the language pretty useless for a more general usage.

from nubasic.

Hello, my answer regarding .bas Source-Files.

I'm not sure what you mean by "switch".

In Python the source code encoding is specified in Line 2 the following way:
#!/usr/bin/python
# -*- coding: iso-8859-1 -*-
or
#!/usr/bin/env python
# -*- coding: utf-8 -*-

It could also be a new Basic command, why put important information into comments ?

An other possibility for having a switch is: The Byte order mark, present = UTF8, not-Present = Latin1 or see my following post.
But does the BOM conflict with the shebang-Function in Linux ???
In a VB.Net- Source File, I found the Byte-Order-Mark ( EF BB BF ) at the beginning.
But VB.Net has no shebang.

Strings

In bigger projects, the language-specific string constants are in "resource" or external files .
Thanks Antonio for writing the Software - one thing at a time, perhaps first do sth regarding the Sourcecode-Question.
There is a "Unicode for C++23" proposal:
https://www.youtube.com/watch?v=3utLG0Qm1Ek
Currently strings are encoding agnostic, "what comes in, goes out", is my experience with the nubasic command Input#
.
As in
https://stackoverflow.com/questions/30277095/whats-the-definition-of-encoding-agnostic
And this was useful for me.

Currently, nubasic strings are 8-bit-sequences! Only the Source Code is treated as 7-bit.

from nubasic.

I assume, around russia they have a huge amount of cp1251 -coded files, etc.
On a more abstract level there are only two kinds of codings relevant today in our world:

1. Some 256-Character encoding, determined by some Environment Variable(s) / locale settings for Editors and the Terminal. "Extended_ASCII" with 8 bits per character.
2. UTF-8.

So a switch could also be between those two possibilities.
7-bit-Ascii is the common subset of both.

from nubasic.