mnh48-minetest / unicodeparser Goto Github PK

View Code? Open in Web Editor NEW

5.0 4.0 3.0 15.27 MB

Unicode Parser is a client-side mod for Minetest by mnh48, it's a temporary solution to chat in unicode characters.

Home Page: https://github.mnh48.com/unicodeparser/

License: MIT License

Lua 33.44% Python 39.16% Shell 27.40%

minetest minetest-client-mod csm codepoints unicode-parser python2-script workaround utf-8 utf8

unicodeparser's Introduction

Unicode Parser

What is Unicode Parser

Unicode Parser unicodeparser is a client-side mod (CSM) for Minetest. It allows you write unicode codepoint in either \uXXXX format or \0xXXXX format then it will convert to real text string of that codepoint and send it out on public chat. Both GUI and CLI now available.

Intended use

This CSM is used as a workaround of the inability to use Input Method Editors (IME) on Minetest, especially for Windows users as I could use IME on Android version of Minetest but not on Windows version.

This CSM could also be used by certain Ubuntu version users who couldn't paste text in Minetest (where the paste text appears as codepoints and not real text).

How to use

As workaround of IME

If you use Windows, run the included exe file under tools folder, or else run the Python2 script file under tools folder, then write the usual text using your IME into the "Input" field, then just enter and the output codepoint will be copied to your clipboard.

In game, to use GUI then send .ug and a formspec will appear asking for input, paste (Ctrl-V) the codepoint and click on "Say". To use CLI then send .uc followed by the unicode codepoint that was copied from the tool earlier.

As workaround of paste problem

The codepoint appeared when you paste into Minetest itself is supposedly already in the format supported by the CSM, so you don't need to run any tools.

In game, to use GUI then send .ug and directly paste in the input, then click "Say". To use CLI then send .uc followed by the unicode codepoint you want to convert.

CSM in action

GUI version and as workaround of IME:

CLI version and as workaround of paste problem:

To do

Include translation support (which seems to be impossible right now because CSM can't access other files)

It does not work!

Chat appear as empty or as boxes

Please change your Minetest font to something that actually have support for language you're using, for example, change it to Noto Sans CJK JP if you're writing Japanese.

It says "Character conversion failed!"

The server is running as ASCII or other non-Unicode locale, there's nothing you can do to fix this.

License

Licensed under The MIT License, see LICENSE.txt.

Changelog

v1.3

Fix for Minetest 5.3.0
Add error message for servers that restricted usage of CSM that sends chat messages

v1.2

The tool now convert the codepoint when you press enter after writing the text, so you don't need to click the button "Convert" anymore, but the button is still there for compatibility reasons.
The tool now copies the converted codepoint to your clipboard when ypu enter or press the "Convert" button, so you don't need to manually copy (Ctrl-C) anymore.
Added link to GitHub at the tool.
The tools now have icon, which is modified from Minetest icon.
Added Windows executable for the tool so that those who don't have Python2 would not need to install it just to use the tool.
Additional thanks to the following people who either tested the Windows executable file or given some advice to me:
- Heathcliff#0001 (Anime Trending Discord server)
- Fidoze#7037 (Anime Trending Discord server)
- Aqo#5414 (A.I. Channel Unofficial Discord server)
- xnamkcor#3740 (A.I. Channel Unofficial Discord server)

(Note: I purposely ask people not from Minetest community to test it because I don't want biased feedback.)

v1.1

Added support for \uXXXX unicode escape in addition to the existing \0xXXXX unicode sequence.
Added command .uc for command line interface, this means those who don't want to use the GUI can now straight away write the unicode escape in chat.
Updated README to reflect the changes.

v1.0

The original initial release.

unicodeparser's People

Contributors

Stargazers

Watchers

Forkers

hagb yquemener uccuhacker

unicodeparser's Issues

About other input boxes of Minetest

This CSM is only for chat boxes. Is it possible to solve unicode inputting in all input boxes of Minetext by a CSM?

there is also \Uhhhhhhhh

I ended up writing this from scratch but still the ones above 0xFFFF are not working.

local sc = string.char
local fl = math.floor
local function utf8_enc(u)
	-- invalid codepoint
	if u >= 0xD800 and u <= 0xDFFF then return '?' end

	-- same
	if u < 0x80 then return sc(u) end

	
	local a, b, c, d
	if u < 0x800 then
		a = 0xC0 + fl(u / 0x40)
		b = 0x80 + (u % 0x40)
		return sc(a, b)
	end

	if u < 0x10000 then
		a = 0xE0 +  fl(u / 0x1000)
		b = 0x80 + (fl(u / 0x40) % 0x40)
		c = 0x80 + (u % 0x40)
		return sc(a, b, c)
	end

	-- 𠀋 \U0002000b becomes \u0000b somewhere after sending
	-- minetest.log still works as expected; shows the character in-game
	if u <= 0x10FFFF then
		a = 0xF0 + fl(u / 0x40000)
		b = 0x80 + (fl(u / 0x1000) % 0x40)
		c = 0x80 + (fl(u / 0x40) % 0x40)
		d = 0x80 + (u % 0x40)
		--return '?'
		minetest.log("error", sc(a, b, c, d))
		return sc(a, b, c, d)
	end

	return '?'
end

core.register_on_sending_chat_messages(function(message)
	message = message:gsub('\\U(%x%x%x%x%x%x%x%x)', function(u)
		--minetest.debug(u)
		return utf8_enc(tonumber(u, 16)) end
	)

	message = message:gsub('\\u(%x%x%x%x)', function(u)
		--minetest.debug(u)
		return utf8_enc(tonumber(u, 16)) end
	)

	-- \0xhhhh or \xhhhh
	message = message:gsub('\\0?x(%x%x%x%x)', function(u)
		--minetest.debug(u)
		return utf8_enc(tonumber(u, 16)) end
	)

	minetest.send_chat_message(message)
	return true
	end
)

-- command .uc for cli version
minetest.register_chatcommand("dummy", {
	params = "<arg>",
	description = "Dummy function",
	func = function(param)
		minetest.send_chat_message("Dummy TEXT")
	end
})

minetest.log('dummy parser loaded')

A error in Minetest 0.4.17.1

I got a error

2018-07-26 00:51:37: ERROR[Main]: ModError: Failed to load and run script from /home/hagb/.minetest/mods/unicodeparser/init.lua:
2018-07-26 00:51:37: ERROR[Main]: /home/hagb/.minetest/mods/unicodeparser/init.lua:44: attempt to call field 'register_on_formspec_input' (a nil value)
2018-07-26 00:51:37: ERROR[Main]: stack traceback:
2018-07-26 00:51:37: ERROR[Main]: /home/hagb/.minetest/mods/unicodeparser/init.lua:44: in main chunk
2018-07-26 00:51:37: ERROR[Main]: Check debug.txt for details.

Thanks!

2019-06-11 21:26:40: ERROR[Main]: ModError: Failed to load and run mod "unicodeparser":
2019-06-11 21:26:40: ERROR[Main]: unicodeparser:init.lua:25: attempt to call field 'register_on_connect' (a nil value)
2019-06-11 21:26:40: ERROR[Main]: stack traceback:
2019-06-11 21:26:40: ERROR[Main]: 	unicodeparser:init.lua:25: in main chunk