Converter should use UTF-8 for source code and generated HTML about es-spec-html HOT 1 CLOSED

I guess my initial reaction is: what problem are we trying to solve with this issue? The output people see on screen is of course identical, equally-correct Unicode either way.

This is going to be long, but just to clarify exactly where I’m coming from, here are the reasons I’m using escape sequences:

• A primary goal of this project is to help TC39 transition to maintaining the spec in HTML rather than Word, if that should ever happen. That is, the HTML output may become the spec source code. I’m going to make sure that I can say, if ever asked, “Sure, the output is just plain ASCII; all non-ASCII characters are escaped; you will not have any charset problems whatsoever.” In both perception and reality this matters. I can’t know what tools a future editor will use. I do know those tools will work just fine on ASCII. Many programs silently misbehave on UTF-8 input, or require extra care when used with UTF-8, particularly on Windows.
• When editing HTML or Python source code, I like to know at a glance whether I’m looking at an ASCII apostrophe or a Unicode right single quotation mark (and other similar fine distinctions). The great thing about Unicode is that it makes it possible to get these fine details right. I don’t bother when I’m writing email, but in a language specification, it really matters. We definitely want the ASCII ' for JS strings and the Unicode ’ for English contractions. So the distinction should be very obvious when looking at the source.
• In practically all editors, non-ASCII characters are hard to type or hard to remember how to type. In VIM, to type ’, you use Ctrl-K ' 9. In Emacs, Ctrl-x 8 ENTER and then type right single quo and hit TAB, then ENTER again. In GUI editors on the Mac, Alt-}. And so on. This is silly. More people know ’.

In short, I’m using escape sequences in order to be kind to human beings who will have to work with this stuff. Which so far is just us, but I’m optimistic.

Incidentally, the program breaks paragraphs into multiple short-ish lines on output—also arguably a silly and rather dated thing to do—for more or less the same reasons.

I don’t see that you’re making any argument in favor of the specific change you’re proposing. For example, you mention that browsers support UTF-8 just fine. True, but they also support character references just fine, so that’s not an argument either way. That seems to be the case for all your bullet points. They are arguments that Unicode and UTF-8 are widely supported (which is uncontroversial), but that’s not the argument you’d need to make: you’d have to argue specifically in favor of UTF-8 over Python escape sequences and HTML character references, since that’s the change you want to make. I think that’s a losing argument and perhaps not the best use of our time...

Thanks for contributing. I appreciate your work.

Cheers,
-j

from es-spec-html.