Fix SIGSEGV on process exit in Qt6 module

[wltechblog]

Issue:

This bug is visible when using kdenlive, when rendering otherwise completes, mlt throws a segfault during exit. This causes kdenlive to display that the process has failed.

Cause:

The QApplication created in createQApplicationIfNeeded() was never deleted, causing a crash during C++ static destruction when Qt6's XCB event queue thread is still running.

Two fixes:

1. Store QApplication in std::unique_ptr for controlled destruction during this DSO's __cxa_finalize, before Qt dependencies are unmapped.

2. Pin the DSO with RTLD_NODELETE to prevent dlclose() from unmapping Qt dependencies while their static destructors are still pending.

Backtrace:
exit() -> __cxa_finalize() -> QObject::~QObject() -> setParent_helper()
-> QCoreApplication::sendEvent() -> SEGFAULT

Tested on GhostBSD 15 with Qt 6.10.2, XCB platform.

[Copilot]

Pull request overview

Fixes a crash on process exit in the Qt6 module by ensuring the Qt application object and its dependent DSOs remain valid through C++ static destruction, avoiding unload-order issues triggered by dlclose().

Changes:

• Pin the Qt module DSO in memory on UNIX using dlopen(..., RTLD_NODELETE) via a constructor hook.
• Store the QApplication created by MLT in a static std::unique_ptr so it is deleted during DSO finalization.

[ddennedy]

If you want to fix the clang-format failure (not strictly necessary)

--- src/modules/qt/common.cpp
+++ src/modules/qt/common.cpp
@@ -18,8 +18,8 @@
 
 #include "common.h"
+#include <memory>
 #include <QApplication>
 #include <QImageReader>
 #include <QLocale>
-#include <memory>
 
 #if defined(Q_OS_UNIX) && !defined(Q_OS_MAC) && !defined(Q_OS_ANDROID)
@@ -42,6 +42,5 @@ __attribute__((constructor)) static void pin_mltqt_in_memory()
     // Clear any stale error before calling dladdr.
     dlerror();
-    if (dladdr(reinterpret_cast<const void *>(pin_mltqt_in_memory), &info)
-        && info.dli_fname) {
+    if (dladdr(reinterpret_cast<const void *>(pin_mltqt_in_memory), &info) && info.dli_fname) {
         void *handle = dlopen(info.dli_fname, RTLD_NOW | RTLD_NODELETE);
         if (!handle) {
@@ -50,6 +49,8 @@ __attribute__((constructor)) static void pin_mltqt_in_memory()
             // when they run during process teardown.  If pinning fails we log a
             // warning — the process *may* crash on exit.
-            mlt_log_warning(NULL, "mltqt: failed to pin DSO (%s) in memory: %s\n",
-                            info.dli_fname, dlerror());
+            mlt_log_warning(NULL,
+                            "mltqt: failed to pin DSO (%s) in memory: %s\n",
+                            info.dli_fname,
+                            dlerror());
         }
     }

[ddennedy]

This is actually causing a crash on exit for me when I never had that problem before, in general:
MLT_REPOSITORY_DENY=libmltopenfx gdb --args melt qtext:

Thread 1 "melt" received signal SIGSEGV, Segmentation fault.
0x00007fffe75e26c4 in QGuiApplication::~QGuiApplication() () from /home/ddennedy/Qt/6.10.1/gcc_64/lib/libQt6Gui.so.6
(gdb) bt
#0  0x00007fffe75e26c4 in QGuiApplication::~QGuiApplication() ()
    at /home/ddennedy/Qt/6.10.1/gcc_64/lib/libQt6Gui.so.6
#1  0x00007fffe8192c9d in QApplication::~QApplication() ()
    at /home/ddennedy/Qt/6.10.1/gcc_64/lib/libQt6Widgets.so.6
#2  0x00007fffe7f8ce38 in std::default_delete<QApplication>::operator()(QApplication*) const
    (this=0x7fffe7fff008 <s_app>, __ptr=0x55555556d110)
    at /usr/include/c++/12/bits/unique_ptr.h:95
#3  0x00007fffe7f8cae0 in std::unique_ptr<QApplication, std::default_delete<QApplication> >::~unique_ptr() (this=0x7fffe7fff008 <s_app>, __in_chrg=<optimized out>)
    at /usr/include/c++/12/bits/unique_ptr.h:396
#4  0x00007ffff7a45495 in __run_exit_handlers
    (status=0, listp=0x7ffff7c1a838 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at ./stdlib/exit.c:113
#5  0x00007ffff7a45610 in __GI_exit (status=<optimized out>) at ./stdlib/exit.c:143
#6  0x00007ffff7a29d97 in __libc_start_call_main
     (main=main@entry=0x555555559d7c <main>, argc=argc@entry=2, argv=argv@entry=0x7fffffffdd48)
    at ../sysdeps/nptl/libc_start_call_main.h:74
#7  0x00007ffff7a29e40 in __libc_start_main_impl
     (main=0x555555559d7c <main>, argc=2, argv=0x7fffffffdd48, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdd38) at ../csu/libc-start.c:392
#8  0x0000555555557c85 in _start ()

Have you tried using a build of MLT from the tip of git master? See #1175. melt now does a hard exit without attempting cleanup that is more trouble than its worth. Your change seems like a good attempt to make a root cause fix, but it is probably no longer worthwhile, but I (and many others) could never reproduce the problem. So, it seems the ugly solution is looking the best.

[wltechblog]

I'll test with the tip. The previous diff may address the issue, although there may be something i'm missing with differences between clang and gcc builds. I'm testing on GhostBSD (based on FreeBSD) currently, which reliably crashes. I haven't had a crash on my Linux systems.

[wltechblog]

The hard exit may have eliminated the crash, i'll do additional testing tomorrow. Thanks!