Support Unicode Bidi-algorithm

[Ghabry]

Text can be mixed direction like e.g. mixing Hebrew and English or numbers:

שדחHallo קח42שמ

Player does not apply bidi here as Harfbuzz does not support such mixed-bidi:

[Ghabry]

Actually hooking in ICU BIDI isn't that hard. Quick proof of concept:

Before:

After:

UErrorCode errorCode = U_ZERO_ERROR;
	UBiDi *bidi = ubidi_open();

	// Set the input text and specify its direction
	auto input = icu_75::UnicodeString::fromUTF32(reinterpret_cast<const UChar32*>(tx.data()), tx.length());

	ubidi_setPara(bidi, input.getBuffer(), -1, UBIDI_DEFAULT_LTR, NULL, &errorCode);

	// Get the visual order of the characters
	int32_t length = ubidi_getProcessedLength(bidi);

	UChar *output = (UChar *)malloc(sizeof(UChar) * (length + 1));

	auto outlen = ubidi_writeReordered(bidi, output, length + 1, UBIDI_DO_MIRRORING, &errorCode);

	icu_75::UnicodeString us(output, outlen);

	std::string s;
	us.toUTF8String(s);

	std::u32string txt = Utils::DecodeUTF32(s);

	free(output);
	ubidi_close(bidi);

[Ghabry]

Was also made aware of that in the descriptions the text reads backwards.

That was due to the "half width space" algorithm. Moved this into the Font class to fix this.

Before (top and bottom text should read the same):

After:

[Ghabry]

there is one more complex problem: When a message contains any command codes or ExFont it will have the same "reverse rendering" issue as the help text.

That must be rewritten to shape the entire line in one go and just assign "attributes" like the color to the individual glyphs. Instead of shaping and rendering everytime a command code is encountered.

[Ghabry]

Glad that all the Harfbuzz shaping stuff was already implemented over multiple weeks. This made the Bidi implementation pretty straightforward (only needed one day)

---

Here a message box with bidirectional text. Each text segment flows in based on its direction (is slowed down to show this better, normal message box is much faster).

simplescreenrecorder-2024-07-08_22.10.18.mp4

Command codes work but I havn't fully tested them yet.

---

@MAZ12211 was so kind to give me a Yume Nikki translation in "the proper way" (how you would type it when the application has proper RTL-Bidi-Support).

Here the current progress (the width calculation is still a bit bugged as you can see from the inconsistent right alignment).

[Ghabry]

Also open for discussion is how to "flip" the Ui.

Obviously full RTL support here needs custom code for the more complex windows but just super-simple Ui flipping could be one function call per window:

[Ghabry]

Pushed the current (incomplete) code here: https://github.com/Ghabry/easyrpg-player/tree/bidi

[MAZ12211]

I just tested the bidi build with Toilet In Wonderland and Ib and both ended up crashing the Player

explorer_8JOtxATMi2.mp4

I'll assume you need the dmp file

[Ghabry]

Actually the dump file doesn't help because I do not have access right now to the computer where I compiled it 😅.

Currently easier is when you provide me with the translation files that crash.

[MAZ12211]

Lol alright
I'll send the language folder because I have no clue what caused it to crash either
Toilet In Wonderland
Ib