lcf2xml - wrong encoding

[SupSuper]

When using the lcf2xml tool, the xml files are tagged with:
<?xml version="1.0" encoding="UTF-8"?>
However RPG Maker does not use UTF-8, and it's not being converted to UTF-8 either. This breaks any XML parser when there are foreign characters in the game. This is pretty easy to test by just putting áéíóú on a map name.

Either the tag should be corrected to match the RPG Maker encoding (probably the Windows default ISO-8859-1) or it should be converted.

[Ghabry]

the encoding detection is bad. Compared to EasyRPG Player there is no autodetection involved. You have to edit the RPG_RT.ini and type in there

[EasyRPG]
Encoding=1252

Then it will work.

[carstene1ns]

Question is, if we ever can do anything about this. We need to guess the encoding, and for that we need data.
For database and some maps this is easy, as there are enough comments/texts/messages or such to feed into the parser.
Other files are not possible, because you have multiple possibilities, false positives and generally overlapping symbols (and we do not talk about translation yet, that have mixed some language).

[SupSuper]

@carstene1ns Since the tools work with individual files, not the whole game, I think it's fine for them to just assume the default Windows encoding (since that's what RPGMaker does) and if that's wrong, then it's on the user to specify a different encoding. Treating Extended ASCII as UTF-8 will always be wrong, most editors will refuse to parse it or irreparably corrupt it.

Basically I'm trying to do some lmu->xml->lmu manipulation. If the xml is invalid UTF-8 it fails. I don't mind if the xml is valid but gibberish, so long as it converts back to the same lcf.

@Ghabry Thanks that works, would be nice to expose that as a command-line option as well.

[Ghabry]

This is solved by now. It uses the system encoding by default and accepts --encoding