UTF-8 "\xF3" does not map to Unicode at /usr/share/perl5/vendor_perl/Locale/Po4a/TransTractor.pm line 583 about po4a HOT 5 CLOSED

I don't see this char in the source code. Help is welcome, please.

it's in spanish addendums, which are in ISO-8859-15 charset. It's ó in Traducción.

It seems they correctly specify the addendum charset in po4a for Spanish, but po4a tries to read it in UTF-8 anyway... so it's probably a po4a's bug.

Besides that po4a probably should print some less cryptic error message with a reference to the file at least and maybe continue with the rest of the documents instead of straight up crashing...

To reproduce:

git clone https://salsa.debian.org/debian/debhelper.git
cd debhelper
po4a man/po4a/po4a.cfg

PS: po4a-translate works fine:

po4a-translate -f pod -m dh_auto_build -l man/es/dh_auto_build.pod -p man/po4a/po/es.po -A ISO-8859-15 -a man/po4a/add3.es

from po4a.

Ok, I think I got it. I must have forgotten to use PerlIO on the filehandle that is in charge of the addendum. I'll look at it later on.

Many many thanks @Fat-Zer for this great diagnostic. You really rock.

from po4a.

I don't see this char in the source code. Help is welcome, please.

from po4a.

Ok, I nailed this down, I guess. The main issue was the po4a.cfg file which is not nice with us.

[po4a_alias:pod] pod opt_fr:"-L ISO-8859-15 -A UTF-8"
[po4a_alias:pod] pod opt_es:"-L UTF-8 -A ISO-8859-15"
[po4a_alias:pod] pod opt_de:"-L ISO-8859-15 -A UTF-8"
[po4a_alias:pod] pod opt_pt:"-L UTF-8 -A UTF-8"
[po4a_alias:pod] pod opt_ja:"-L UTF-8 -A UTF-8"

See how it's redefining the pod alias several times. I think that this was already broken in the past (only the last definition is kept) but before the introduction of PerlIO, the -A ISO-8859-15 was useless for some reason involving Perl trying to magically guess what to do.

I fixed this in commit 300208e197ce4e8a7692ed835f093abb8c07007b, but debhelper still fails with this error:

Malformed encoding while writing char '–' to file /tmp/debhelper/man/de/debhelper.pod with charset ISO-8859-15: "\x{2013}" does not map to iso-8859-15 at 

This is because of the -L iso-8859-15 in the config file, which shouldn't be there since most languages of the list use UTF-8 chars that cannot be represented in Latin1. It used to work because all those -L parameters where wrongly discarded because of the previous bug. I guess that if you move the opt_fr line after the opt_ja one, it will fail as it is right now, because in that case, the request to write french files in Latin1 is not ignored anymore, and it fails because the french translation contains …, the char representing 3 dots, that cannot be represented in Latin1.

I think that my code fixes this bug, but I leave it open for further discussion in case someone has a better idea.

from po4a.

Ok, nobody spoke out, so I'm closing the bug. Feel free to reopen on need.

from po4a.