UnicodeEncodeError: 'utf-8' codec can't encode character '\udcc3' about moodle-dl HOT 8 CLOSED

Ah thank you very much! This is very helpful. I'll try to prevent it.

from moodle-dl.

Looks like your Moodle has at least one invalid UTF-8 text which the downloader tries to store in the local database. I will think of a solution tomorrow. Either allow invalid UTF-8 or remove the affected characters, not sure which would be better.

from moodle-dl.

Was this the first time you run the downloader?

from moodle-dl.

It was the first time I tried to download all the Moodle courses.
I tried a lot today and I think the error was not caused by a Moodle entry with invalid UTF-8 characters.
The error seems to correlate with pressing arrow keys during renaming of courses.
I renamed all the courses in the config dialog and made a mistake.
I tried to use the arrow keys to navigate to the place of the typo.
This did not work and I deleted the name and rewrote it.
My guess is that a keystroke is still in the configuration.
So it might be useful to prevent such entries in the configuration dialog.

from moodle-dl.

Do you use Linux?

from moodle-dl.

I use the python:3 docker image which is based on Debian.

from moodle-dl.

Ah found the problem. Arrow keys should now working in input. But i can not prevent some complete strange utf-8 input, I have no idea how i could do this without dropping utf-8 support completely.
There is a way to adjust the SQL-Lite cursor so that it drops invalid utf-8 characters, but for now this problem should be fixed. If such a problem occurs again I will take harder steps.

I'm gonna push this to main tonight.

from moodle-dl.

Thank you! I really appreciate your efforts.

from moodle-dl.