no subconstruct matched: TextIns <livemaker.lsb.novel.TpWord object at 0x000001F8727A09D0>

[zhumating]

• pylivemaker version:1.0.1
• Python version:3.8
• Operating System: windows10
• LiveMaker game: a game made by LiveMaker2

Description

I'm translating a LiveMaker game from Japanese to Chinese.
I read #14 and changed some encode/decode code from cp932 to

try:
    cp936
except:
    cp932

For most .lsb file, it works. But at one it failed and I got this:

For this .lsb file, I translated its text to Chinese at first(at that time the text is correct), and then, I translated some menu by lmlsb edit.But after changed menu, I find that text is incorrect. So I tried to batchinsert .lsb file again but I failed.
The .lsb file which was failed is here:
00000001.lsb.zip
And translated .lns files are here:
00000001.lns.zip

What I Did

lmlsb batchinsert 00000001.lsb 00000001
Backing up original LSB.
Scenario 000003BC at line 8 will be replaced.
Scenario 000003BD at line 14 will be replaced.
Scenario 000003F6 at line 26 will be replaced.
Scenario 000003F7 at line 32 will be replaced.
Scenario 000003F8 at line 38 will be replaced.
Scenario 000003F5 at line 20 will be replaced.
Scenario 000003F9 at line 44 will be replaced.
Scenario 000003FB at line 85 will be replaced.
Scenario 0000044C at line 346 will be replaced.
Scenario 000004B0 at line 651 will be replaced.
Scenario 00000451 at line 380 will be replaced.
Scenario 00000450 at line 374 will be replaced.
Scenario 00000407 at line 213 will be replaced.
Scenario 000003FC at line 91 will be replaced.
Scenario 000003FF at line 136 will be replaced.
Scenario 00000400 at line 142 will be replaced.
Scenario 00000402 at line 183 will be replaced.
Scenario 00000403 at line 189 will be replaced.
Scenario 00000404 at line 195 will be replaced.
Scenario 00000405 at line 201 will be replaced.
Scenario 00000406 at line 207 will be replaced.
Scenario 00000409 at line 254 will be replaced.
Scenario 0000040B at line 295 will be replaced.
Scenario  at line 299 will be replaced.
Scenario 0000040C at line 305 will be replaced.
Scenario 0000044D at line 352 will be replaced.
Scenario 0000044E at line 358 will be replaced.
Scenario  at line 362 will be replaced.
Scenario  at line 366 will be replaced.
Scenario 00000455 at line 491 will be replaced.
Scenario 00000456 at line 497 will be replaced.
Scenario 00000457 at line 503 will be replaced.
Scenario  at line 507 will be replaced.
Scenario 00000458 at line 513 will be replaced.
Scenario 0000045A at line 558 will be replaced.
Scenario 0000045B at line 564 will be replaced.
Scenario 0000045C at line 570 will be replaced.
Scenario 0000047E at line 579 will be replaced.
2020-11-26 17:43:55.047 | WARNING  | livemaker.lsb.novel:compile:1315 - Attempting to compile script which was not generated by pylivemaker.
Scenario 0000047F at line 585 will be replaced.
Scenario 00000480 at line 591 will be replaced.
Scenario 00000481 at line 597 will be replaced.
Scenario 00000482 at line 603 will be replaced.
Scenario 00000484 at line 609 will be replaced.
Scenario 0000048D at line 638 will be replaced.
Scenario 0000048E at line 644 will be replaced.
Scenario 000004C1 at line 658 will be replaced.
Scenario 000004C2 at line 664 will be replaced.
Scenario 000004C7 at line 670 will be replaced.
Could not generate new LSB file: Error in path (building) -> commands -> items
no subconstruct matched: TextIns <livemaker.lsb.novel.TpWord object at 0x0000022D9FED99D0> "メッセージボックス" 1 0 0

[pmrowla]

Changing the locale isn't really supported, as the LM engine is hard coded to only work with CP932.

It is technically possible to get some alternate encodings to work, but it requires reverse engineering your specific game's engine exe to find the right constants and changing them to the new encoding. If your game was made with the older livemaker 2 engine, it may not be possible at all. With regard to what was being done in #14, we basically only got it to work at all in the most recent (final) version of LM3.

[zhumating]

At first time I lmpatch .lsb file into .ext file, the text is displayed correctly. And I tried to edit the menu of original .lsb. After dump, I find the original text is correct, but some "メッセージボックス" that I don't want to change changed to 儊僢僙乕僕儃僢僋僗.
So I guess the key point is at the lsb\core.py, class Param(BaseSerializable), _struct method.
I tried to change class StringEncoded(at construct\core.py), but I failed. If I put cp932 at first, the translate menu can not be read. And if I put cp936 at first, the "メッセージボックス" I don't want to change can not be read. So I think I shouldn't use except UnicodeEecodeError, but to use range of cp932(like if ch >= 'A' and ch <='Z'). But I have little knowledge of cp932. I'll try to find the list of cp932 and try again.

[zhumating]

I changed some code in construct\core.py StringEncoded class:

    def _decode(self, obj, context, path):
        det = chardet.detect(obj)
        encoding = det["encoding"]
        if encoding != None:
            encoding = encoding.lower()
        encodings = ["cp932","SHIFT_JIS","ascii",None]
        if encoding in encodings:
            ch = obj.decode(self.encoding)
        else:
            ch = obj.decode("cp936")
        return ch

    def _encode(self, obj, context, path):
        if not isinstance(obj, unicodestringtype):
            raise StringError("string encoding failed, expected unicode string", path=path)
        if obj == u"":
            return b""
        try:
            ch = obj.encode(self.encoding)
        except:
            ch = obj.encode("cp936")
        return ch

And this time I get correct menu, part of correct text and part of incorrect text...and sometimes namelabel fly to other place

Maybe I should change some reference of PascalString? But I have no idea where should change...

[zhumating]

After I changed _encode in livemaker\lsb\novel.py _TWdCharAdapter class, I successed.

def _encode(self, obj, ctx, path):
        try:
            result = int.from_bytes(obj.encode("cp936"), byteorder="big")
            return result
        except UnicodeEncodeError:
            return int.from_bytes(obj.encode("cp932"), byteorder="big")

I edited menu first, and then I lmlsb batchinsert the text. Actually, there's still something wrong in my code(word like "不借" can not be displayed correctly), but I decide to change these words to other same meaning words.
And I don't know what will happen if I batchinsert at first.

[pmrowla]

It should be possible to do without editing construct.

The reason it will error out when you try to translate menus (without the edited construct/core.py) is because menu strings are packed as string variable values in:

pylivemaker/livemaker/lsb/core.py

Line 409 in 35ea5eb

class Param(BaseSerializable):

To convert menu items to a different codepage, you need to edit the struct for that class to support the new encoding

In:

pylivemaker/livemaker/lsb/core.py

Lines 477 to 499 in 35ea5eb

	@classmethod
	def _struct(cls):
	return construct.Struct(
	"type" / construct.Enum(construct.Byte, ParamType),
	"value"
	/ construct.Switch(
	construct.this.type,
	{
	"Int": construct.Int32sl,
	"Float": construct.ExprAdapter(
	construct.Bytes(10),
	lambda obj, ctx: numpy.frombuffer(obj.rjust(16, b"\x00"), dtype=numpy.longdouble),
	lambda obj, ctx: numpy.longdouble(obj).tobytes()[-10:],
	),
	"Flag": construct.Byte,
	"Str": construct.PascalString(construct.Int32ul, "cp932"),
	},
	# else 'Var' variable name type
	construct.Select(
	construct.PascalString(construct.Int32ul, "cp932"),
	),
	),
	)

The line

                    "Str": construct.PascalString(construct.Int32ul, "cp932"),

should be changed to something like

                    "Str": construct.Union(
                        construct.PascalString(construct.Int32ul, "cp936"),
                        construct.PascalString(construct.Int32ul, "cp932"),
                    ),

which will tell pylm to try to pack string variables as either CP936 first, and then fall back to CP932.

(You should not need to change the "Var" encoding unless you have been translating the actual system variable names via lmlsb edit for some reason).

[zhumating]

I installed construct again and changed line 492 of livemaker/lsb/core.py to

                    "Str": construct.Union(
                        construct.PascalString(construct.Int32ul, "cp936"),
                        construct.PascalString(construct.Int32ul, "cp932"),
                    ),

then I tried

lmlsb edit 000000CD.lsb 66

and I got a construct.core.UnionError:

lmlsb edit --help
Traceback (most recent call last):
File "c:\python38\lib\runpy.py", line 193, in run_module_as_main
return run_code(code, main_globals, None,
File "c:\python38\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Python38\Scripts\lmlsb.exe_main.py", line 4, in
File "c:\python38\lib\site-packages\livemaker_init.py", line 30, in
from .lsb import LMScript, LNSCompiler, LNSDecompiler
File "c:\python38\lib\site-packages\livemaker\lsb_init.py", line 21, in
from .lmscript import LMScript
File "c:\python38\lib\site-packages\livemaker\lsb\lmscript.py", line 41, in
from .command import CommandType, PropertyType, command_classes, command_structs
File "c:\python38\lib\site-packages\livemaker\lsb\command.py", line 31, in
from .novel import TpWord
File "c:\python38\lib\site-packages\livemaker\lsb\novel.py", line 478, in
class TWdOpeVar(BaseTWdGlyph):
File "c:\python38\lib\site-packages\livemaker\lsb\novel.py", line 500, in TWdOpeVar
"var_name_params" / construct.If(construct.this...version < 102, LiveParser._struct()),
File "c:\python38\lib\site-packages\livemaker\lsb\core.py", line 829, in _struct
"entries" / construct.PrefixedArray(construct.Int32ul, OpeData._struct()),
File "c:\python38\lib\site-packages\livemaker\lsb\core.py", line 584, in _struct
"operands" / construct.Array(construct.this.count, Param._struct()),
File "c:\python38\lib\site-packages\livemaker\lsb\core.py", line 492, in _struct
"Str": construct.Union(
File "c:\python38\lib\site-packages\construct\core.py", line 3354, in init
raise UnionError("parsefrom should be either: None int str context-function")
construct.core.UnionError: parsefrom should be either: None int str context-function

Is there anything wrong when I copy the code? Or should I update my construct to 2.10?

[pmrowla]

Hmm, it might be that construct union doesnt work properly with pascalstring, and that to get it to work you'd need to read the length field and then union byte arrays for each encoding.

I don't really have time to investigate/test this issue right now, if editing construct itself works for you it may be better to just go back to doing that for now.

[Stefan311]

I have successfully encoded localized menu. see #14 from 17 may

[zhumating]

Hmm, it might be that construct union doesnt work properly with pascalstring, and that to get it to work you'd need to read the length field and then union byte arrays for each encoding.
I don't really have time to investigate/test this issue right now, if editing construct itself works for you it may be better to just go back to doing that for now.

Thank you very much!

I have successfully encoded localized menu. see #14 from 17 may

Thank you for your advice! I'll try it.

[Xer0-Faust]

Hmm, it might be that construct union doesnt work properly with pascalstring, and that to get it to work you'd need to read the length field and then union byte arrays for each encoding.
I don't really have time to investigate/test this issue right now, if editing construct itself works for you it may be better to just go back to doing that for now.

Thank you very much!

I have successfully encoded localized menu. see #14 from 17 may

Thank you for your advice! I'll try it.

I am trying to translate some LM3 based game to Cp936 , it worked on LM2 but not on LM3.
Do you solve the problem?

[zhumating]

Hmm, it might be that construct union doesnt work properly with pascalstring, and that to get it to work you'd need to read the length field and then union byte arrays for each encoding.
I don't really have time to investigate/test this issue right now, if editing construct itself works for you it may be better to just go back to doing that for now.

Thank you very much!

I have successfully encoded localized menu. see #14 from 17 may

Thank you for your advice! I'll try it.

I am trying to translate some LM3 based game to Cp936 , it worked on LM2 but not on LM3.
Do you solve the problem?

Yes,because of help of Stefan311, I solved the problem. But my English is terrible, I can't describe it clearly. Do you know Chinese? Could we talk by Chinese by E-mail？

[Xer0-Faust]

Hmm, it might be that construct union doesnt work properly with pascalstring, and that to get it to work you'd need to read the length field and then union byte arrays for each encoding.
I don't really have time to investigate/test this issue right now, if editing construct itself works for you it may be better to just go back to doing that for now.

Thank you very much!

I have successfully encoded localized menu. see #14 from 17 may

Thank you for your advice! I'll try it.

I am trying to translate some LM3 based game to Cp936 , it worked on LM2 but not on LM3.
Do you solve the problem?

Yes,because of help of Stefan311, I solved the problem. But my English is terrible, I can't describe it clearly. Do you know Chinese? Could we talk by Chinese by E-mail？

好的

[yurilove]

Hmm, it might be that construct union doesnt work properly with pascalstring, and that to get it to work you'd need to read the length field and then union byte arrays for each encoding.
I don't really have time to investigate/test this issue right now, if editing construct itself works for you it may be better to just go back to doing that for now.

Thank you very much!

I have successfully encoded localized menu. see #14 from 17 may

Thank you for your advice! I'll try it.

I am trying to translate some LM3 based game to Cp936 , it worked on LM2 but not on LM3.
Do you solve the problem?

Yes,because of help of Stefan311, I solved the problem. But my English is terrible, I can't describe it clearly. Do you know Chinese? Could we talk by Chinese by E-mail？

您好，请问您有成功汉化菜单和选项吗？我目前也在尝试汉化，所以想请教下您