[Feature] Song subtitles for the OP cinematic

[ThePlayer14]

The OP cinematic doesn't have subtitles for the song Ayane (彩音) - Nageki no mori (嘆きノ森).

I think it could be added onto the cinematic either via injection of Substation subtitle (.ass) as an overlay, using either libass or VSFilter to render the Substation file on top of the cinematic.

Another approach would be the creation of hard-subbed cinematics, and have a setting for them (just like the BGM / SE toggles) to set them accordingly.

I have a test file that's likely good, but I'm not sure if the Eng translation is accurate since it's a test run translation, but the timing seems good.

higu-gm-sub_v2.ass.txt

(to use this, remove the .txt extension first.)

Hopefully it can be done.

[drojf]

I'll just preface this by saying this probably won't get fixed. But if it were to be fixed:

Side-note: Other chapters

I think none of the other chapters have subtitles in their videos. It would make sense to add subtitles all of them, rather than just one chapter.

Translation Quality

We would only accept subtitles if we were sure it were properly translated. Some people on our team can check translations, but if you used machine translation without knowing Japanese at all, we might not accept it, for example.

Option 1: Hard Subs

Currently we have two video files (for most of the chapters) to support both windows and linux/macos. So if we were to have an option for subtitles, we would need four copies of each video.

Otherwise, we could also make having subtitles the default, and keep the same number of videos.

Option 2: Soft subs / text overlay

Via existing engine only

Possibly I could use the existing engine to overlay text ontop of the video as a rudimentary subtitle system.

Via external library or upgrading the video player

In the past, someone (I think a translator) asked whether I could add the UMP (I think "it's called "Universal Media Player" based on VLC, but it's no longer supported) to have different video format and soft subs. Surprisingly, I got it working (as a test), but I essentially had to add source code from an unknown source the person supplied me. That was on this branch: https://github.com/drojf/higurashi-assembly-2/tree/f-ump-video

However, I don't think I wouldn't add it to our main mod, because I couldn't even obtain the source code from an official source.

Summary

Probably the route I would choose (if I were to do it) would be adding soft subs via the engine only, but there would be some investigation work to see if that were even possible. However, at this stage of the mods development, it would be highly unlikely I would work on it unfortunately.

[ThePlayer14]

Thanks for the reply.

I think setting it in the installer to use subbed cinematics could also work, so it will place subbed OPs in the movies folder.

As for the test file, the Romaji part was made using a MeCab wrapper, listening to the vocals, and a dictionary to have a proper result.

The Eng sub part, which is a test feature, is made using hybrid translation using several providers' results then making a common normalized result. This part is very experimental and might not be accurate.

For a release version, I'd do the original + Romaji transcription, which then could be translated by a translator, since it would be only just copying the timing of the Romaji segment and replacing the text in the newly created segment.

[drojf]

Ah, I didn't think about setting it in the installer - that would work, but on the developer side we'd still need to manage four versions of the opening video.

---

Out of curiosity, I tried showing the text window of the game ontop of the video (as a test, I made it print the current video time in milliseconds), which did work.

So it would be possible to display subtitles this way, but of course you lose all of the special subtitle features (unless you implement them in the engine). But I would imagine basic subtitles would be OK. The font would also have to be one of the fonts already embedded in the game (unless you put in extra effort).

I think the library the game uses to display fonts would allow you to do centered, text wrapped text easily, and change the font size, but it's not shown in the below screenshot.

Edit: It's on this branch, but there is heaps of changes which are unnecessary that I added while messing around f-video-subtitles-WIP

[ThePlayer14]

So, I've tried to do some testing with either the mp4 and the ogv variants, re-encoded with subs and one of them did succeed:

Not sure which one it was, but the one I did tried was ffmpeg -i mv01.ogv -codec:v libtheora -qscale:v 7 -acodec copy -vf "ass=higu-gm-sub_v3_test2.ass" mv01_mod.ogv (for the ogv one), for mp4 I tried using avidemux. I've used Arial for the subs.

[ThePlayer14]

Here's the test file I used in this run:

higu-gm-sub_v3_test2.ass.txt