Skip to content

Handle games whose frame number freezes or restarts #436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Remleo wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Handle games whose frame number freezes or restarts #436

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions src/mods/VR.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -550,6 +550,25 @@ class VR : public Mod {
return m_native_stereo_fix_same_pass->value();
}

// Called when the game's frame number exceeds the highest seen, which is the only reliable sign of a
// new frame. Why the highest and not the previous one: begin_render_viewfamily.
void notify_game_frame_advanced() {
m_last_game_frame_advance = std::chrono::steady_clock::now();
}

// True while the game has stopped producing frames, which some games do while a fullscreen menu is up.
//
// Answered here, at the point of use, rather than precomputed where the number is published: if the
// game stops running that path altogether, a stored flag would be stuck at its last value.
//
// Some threshold is unavoidable, because "produced no frame" and "produces frames more slowly than we
// present" differ only in duration, and time is the steadiest unit for it -- a count of presents would
// depend on the game's frame rate against the headset's, a count of calls on how many view families
// the game pushes per frame. 250ms outlasts any plausible slow frame and is invisible when a menu opens.
bool is_game_frame_stalled() const {
return (std::chrono::steady_clock::now() - m_last_game_frame_advance) >= std::chrono::milliseconds(250);
}

bool is_ahud_compatibility_enabled() const {
return m_compatibility_ahud->value();
}
Expand Down Expand Up @@ -1136,6 +1155,9 @@ class VR : public Mod {
bool m_has_hw_scheduling{false}; // hardware accelerated GPU scheduling
bool m_spoofed_gamepad_connection{false};
bool m_aim_temp_disabled{false};
// Written where the view family begins, read while presenting. Unsynchronized on purpose, like the
// timestamps above it: a torn read costs one frame sourcing the wrong eye texture.
std::chrono::steady_clock::time_point m_last_game_frame_advance{std::chrono::steady_clock::now()};

struct {
bool draw{false};
Expand Down
14 changes: 13 additions & 1 deletion src/mods/vr/D3D12Component.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,8 +184,20 @@ vr::EVRCompositorError D3D12Component::on_frame(VR* vr) {
.back = 1
};

// A game that has stopped advancing its frame number has stopped drawing the world into its own
// backbuffer, while the scene capture feeding the right eye keeps rendering one:
//
// left eye backbuffer -- menu only
// right eye scene capture -- still the world
//
// Measured symptom: a frozen world in the right eye against a black left one, which reads as the
// image rotating the wrong way when the head turns. Both eyes from the backbuffer while stalled,
// so both show whatever the game is actually presenting.
auto* const right_source = vr->is_game_frame_stalled() ? m_game_tex.texture.Get()
: m_scene_capture_tex.texture.Get();

commands.copy_region_stereo(
m_game_tex.texture.Get(), m_scene_capture_tex.texture.Get(), render_target,
m_game_tex.texture.Get(), right_source, render_target,
&left_src_box, &left_src_box,
0, 0, 0, m_backbuffer_size[0] / 2, 0, 0,
D3D12_RESOURCE_STATE_RENDER_TARGET,
Expand Down
126 changes: 123 additions & 3 deletions src/mods/vr/FFakeStereoRenderingHook.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2596,6 +2596,12 @@ struct SceneViewExtensionAnalyzer {
static inline uint32_t pre_render_viewfamily_renderthread_index{0};
static inline uint32_t frame_count_offset{0};

// What turns the game's frame number into the key UEVR gives its pose pipeline. Maintained in
// begin_render_viewfamily on the game thread, added by everything that publishes a frame number, so
// the two publishers cannot disagree. Read from the render thread; relaxed because the cost of
// reading it a frame late is a frame, and the alternative is a lock on the hot path.
static inline std::atomic<uint32_t> frame_key_offset{};

template<int N>
static bool analysis_dummy_stage1(ISceneViewExtension* extension, uintptr_t a2, uintptr_t a3, uintptr_t a4) {
if (N == 0) {
Expand Down Expand Up @@ -3464,8 +3470,118 @@ void FFakeStereoRenderingHook::begin_render_viewfamily(ISceneViewExtension* exte

//vr->update_hmd_state(true, frame_count);
auto runtime = vr->get_runtime();
runtime->internal_frame_count = frame_count;
runtime->on_pre_render_game_thread(frame_count);

// The key UEVR gives its pose pipeline has to move forward, and only forward.
//
// Poses live in pipeline_states[key % QUEUE_SIZE], and OpenXR::on_pre_render_game_thread refreshes a
// slot only when it is empty or when the key jumped FORWARD past it. Any other revisit keeps the
// predicted display time already in the slot, so xrLocateViews predicts for a moment that has
// passed. On screen the image comes loose and swims against the head, faster the staler it is.
//
// The game's own number is not that key: it freezes while some games are paused, and it restarts
// from scratch on a level load. Both revisit slots out of order.
//
// number grows key grows with it, offset untouched
// number freezes key keeps moving on its own, so the slots keep rotating
// number restarts key carries on one above the highest it reached
// number repeats key stays put -- it is still the same frame
//
// Kept as an offset rather than a counter of our own, so a game that behaves stays on its own
// numbering and the offset stays zero.
//
// Progress is "exceeded the highest number seen", not "differs from the previous one". A game can
// push several view families per frame, and that breaks an equality test both ways:
//
// repeated numbers passes during ordinary play, shifting the key a frame every frame; the view
// answers head movement with amplified jitter. Seen in Gylt and Silent Hill 2
// alternating almost never passes, so a real stall goes unnoticed
//
// Which one a game shows is up to the game, so a single title proves nothing either way.

// A restart drops the number by orders of magnitude; extra view families differ by a few.
constexpr uint32_t restart_slack = 64;

static uint32_t highest_game_frame_count{};
static uint32_t key_carry{};

if (frame_count > highest_game_frame_count) {
highest_game_frame_count = frame_count;
vr->notify_game_frame_advanced();
} else if (frame_count + restart_slack < highest_game_frame_count) {
// One above the highest, so the key never falls back to the game's new number.
key_carry += highest_game_frame_count + 1 - frame_count;

SPDLOG_INFO("Game restarted its frame numbering ({} after a high of {}), carrying the key on to {}",
frame_count, highest_game_frame_count, frame_count + key_carry);

highest_game_frame_count = frame_count;
vr->notify_game_frame_advanced();
}

// Report which pattern the game submits, once each, because it is not visible any other way.
//
// A repeat only counts while the run stays short. A stalled game repeats its number on every call,
// so an unbounded test would report every stall as the very pattern it has to be told apart from:
// several view families make a run of several, a stall makes a run of hundreds. A restart is
// excluded from the alternating case for the same reason -- it also arrives below the previous
// number, and it is handled above.
constexpr uint32_t max_families_per_frame = 4;

static uint32_t previous_frame_count{};
static uint32_t repeat_run{};

if (frame_count == previous_frame_count) {
++repeat_run;
} else {
if (frame_count > previous_frame_count) {
if (repeat_run > 0 && repeat_run <= max_families_per_frame) {
SPDLOG_INFO_ONCE("Game submits repeated frame numbers within a frame (saw {} {} times in a row)",
previous_frame_count, repeat_run + 1);
}
} else if (previous_frame_count - frame_count <= restart_slack) {
SPDLOG_INFO_ONCE("Game alternates frame numbers per frame ({} after {})", frame_count, previous_frame_count);
}

repeat_run = 0;
}

previous_frame_count = frame_count;

// When a stall ends its frames are folded into the carry, not dropped: handing the key back is the
// same backwards jump a restart makes, and measured stalls reach 159 frames.
static uint32_t stall_frames{};
static bool was_stalled{};

const auto stalled = vr->is_game_frame_stalled();
const auto stalled_for = stall_frames;

if (stalled) {
++stall_frames;
} else if (stall_frames != 0) {
key_carry += stall_frames;
stall_frames = 0;
}

// Both edges, once per stall: the game keeps rendering, so nothing else in the log changes when its
// number freezes. The key is reported with them, so a log alone shows whether it went backwards.
if (stalled != was_stalled) {
was_stalled = stalled;

if (stalled) {
SPDLOG_INFO("Game stopped advancing its frame number, holding {}", frame_count);
} else {
SPDLOG_INFO("Game resumed advancing its frame number at {}, after {} frames of stall, key at {}",
frame_count, stalled_for, frame_count + key_carry);
}
}

const auto effective_frame_count = frame_count + key_carry + stall_frames;

// Published for the render thread, which adds it to the same game frame number.
SceneViewExtensionAnalyzer::frame_key_offset.store(key_carry + stall_frames, std::memory_order_relaxed);

runtime->internal_frame_count = effective_frame_count;
runtime->on_pre_render_game_thread(effective_frame_count);

// This is a HACKHACKHACK to get splitscreen working on around 4.20 to 4.27 something
// This is completely borked on UE5
Expand Down Expand Up @@ -3695,7 +3811,11 @@ void FFakeStereoRenderingHook::pre_render_viewfamily_renderthread(ISceneViewExte
cmd_list = *(sdk::FRHICommandListBase**)((uintptr_t)cmd_list + ue5_command_offset);
}

const auto compensation = g_hook->get_frame_delay_compensation();
// The same offset the game thread applied, so both publishers name a frame the same way. Without it
// the two disagree by the width of a stall or of a numbering restart, and a pose stored under one key
// is looked up under another.
const auto key_offset = (int32_t)SceneViewExtensionAnalyzer::frame_key_offset.load(std::memory_order_relaxed);
const auto compensation = g_hook->get_frame_delay_compensation() + key_offset;

// Using slate's draw window hook is the safest way to do this without
// false positives on the command list in this function
Expand Down
Loading