Skip to content

libobs/util: Prevent locking mutex in child process when checking Qt5 #12018

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 25, 2025

Conversation

norihiro
Copy link
Contributor

@norihiro norihiro commented Mar 31, 2025

Description

In has_qt5_dependency, revert the log handler to the default just after the fork.

This change assumes that base_set_log_handler and def_log_handler do not use any locks.

Motivation and Context

The frontend has set a log handler which locks a mutex in LogString.

If os_dlopen fails in the child process, it calls blog and attempts to lock the mutex that another thread may have already locked before fork().

Though I never encountered the deadlock, it's better to fix it.

How Has This Been Tested?

OS: Fedora 40

Added a plugin binary file which causes an error during os_dlopen.
Ran OBS and checked the log file contains only one error message of the plugin.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • My code has been run through clang-format.
  • I have read the contributing document.
  • My code is not on the master branch.
  • The code has been tested.
  • All commit messages are properly formatted and commits squashed where appropriate.
  • I have included updates to all appropriate documentation.

The frontend has set a log handler which locks the mutex in `LogString`.
If `os_dlopen` fails in the child process, it calls `blog` and attempted
to lock the mutex that may have already been locked by another thread
before `fork()`. This change prevents the child process from locking the
mutex, resolving the potential deadlock.
Copy link
Collaborator

@kkartaltepe kkartaltepe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, though I was thinking we might be able to remove this for the next major version too.

@norihiro
Copy link
Contributor Author

norihiro commented Apr 2, 2025

Approved, though I was thinking we might be able to remove this for the next major version too.

I think so, too. It's much better to remove the Qt5 check in the next major release or the next.

@WizardCM WizardCM added the Bug Fix Non-breaking change which fixes an issue label Apr 6, 2025
@RytoEX RytoEX added this to the OBS Studio 31.1 milestone Apr 25, 2025
@RytoEX RytoEX merged commit 5c44845 into obsproject:master Apr 25, 2025
15 checks passed
@norihiro norihiro deleted the has_qt5_dependency-deadlock branch April 26, 2025 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Fix Non-breaking change which fixes an issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants