Wayland: render drag preview and overlays as child widgets instead of top-level Qt::Tool

[facontidavide]

First of all, thanks a lot for your software, I have been using it for years in Plotjuggler ❤️

While testing it on ubuntu 26.04, i found some problems that this PR is pparently fixing.
Sorry for the AI generated description that follows.

Summary

Under native Wayland, the drag preview drifts off the cursor and drop indicators render over the wrong dock area. This is because CFloatingDragPreview, CDockOverlay, and CDockOverlayCross are all top-level Qt::Tool windows that the client repositions every mouse-move via move(globalCoords). Wayland's xdg_toplevel protocol has no client-driven "move to (x,y)" verb — the compositor owns top-level placement — so those move() calls are silently lossy.

This PR makes the three widgets child widgets of the dock manager's top-level window. Positioning happens inside the parent's surface via parentWidget()->mapFromGlobal(...), which Qt translates correctly on every backend (xcb, Wayland, Windows, macOS).

Why not just Qt::ToolTip (PR #789)?

PR #789 takes a different approach: keep the widgets as top-levels, swap the window flag from Qt::Tool to Qt::ToolTip so Wayland uses xdg_popup (which is client-positionable). It's a smaller patch, but the PR author themselves notes on 2025-11-29:

Floating window can create successfully. But I move the floating window, the dock overlay cannot work properly and I have no idea about it.

The overlay positioning issue is the load-bearing visible bug. Qt::ToolTip partially helps the floating preview but still leaves the overlay broken. The child-widget approach in this PR addresses both with one architectural decision: stop being top-level when you need parent-relative positioning that always honors client requests.

PR #789 and this PR are mutually exclusive — pick one, not both.

What this PR changes

src/FloatingDragPreview.cpp:

• Constructor: in the frameless config (!DragPreviewHasWindowFrame), parent to parent->window() and drop the Qt::Tool | Qt::FramelessWindowHint flag setup. Add WA_TransparentForMouseEvents so a release directly on a visible drop indicator is no longer intercepted by the translucent preview rectangle.
• moveFloating(): convert the screen-coord target to parent-local via mapFromGlobal() when the widget is a child. The legacy top-level path (isWindow() || !parentWidget()) is preserved for DragPreviewHasWindowFrame=true.
• startFloating(): raise() after show() so the child preview paints above sibling docks.

src/DockOverlay.cpp:

• CDockOverlay and CDockOverlayCross constructors: parent to parent->window(), drop the Qt::Tool | FramelessWindowHint | WindowStaysOnTopHint | X11BypassWindowManagerHint block (only meaningful for top-levels), add WA_TransparentForMouseEvents.
• CDockOverlay::showOverlay(): convert the target's top-left to parent-local before move(). raise() after show().
• CDockOverlayCross::updatePosition(): simplified — both widgets share the same parent now, so the offset math is straightforward. raise() after move.
• CDockOverlayCross::updateOverlayIcons(): replace windowHandle()->devicePixelRatio() with devicePixelRatioF(). windowHandle() returns nullptr for a child widget and the original code segfaults as soon as the cross attempts to refresh icon DPR after reparenting.

What this PR does NOT change

• The DragPreviewHasWindowFrame=true config (Windows-style framed preview) keeps the original top-level Qt::Window path. The cross-platform behavior on Windows is untouched.
• X11 / xcb behavior: still works. The X11-specific WindowStaysOnTopHint | X11BypassWindowManagerHint block on Q_OS_UNIX was tied to the top-level Qt::Tool window; with no top-level there's nothing to bypass.
• The drag state machine (mousePress / mouseMove / mouseRelease flow through DockAreaTitleBar, DockWidgetTab, etc.) is untouched.
• The drop indicator label widgets (small icons inside the cross) are untouched — they're now children of a child widget that itself works correctly.

Forecloses two known follow-ups from the #789 thread

• "Can't drag widget out of screen with Qt::ToolTip" (cristianadam, fix docking drag drop problem on Linux wayland #789): doesn't apply — we're not using Qt::ToolTip, and the preview's bounds are intentionally limited to the manager window for now (the typical rearrangement target is inside the app anyway).
• "Need to remove WindowStaysOnTopHint for Wayland" (Wing-summer, fix docking drag drop problem on Linux wayland #789): doesn't apply — the StaysOnTop hint is only relevant for top-levels, and we're not top-level in this code path.

Verification

A standalone reproducer is available at PlotJuggler/PJ4 (https://github.com/PlotJuggler/PJ4/tree/feature/m2-plot-canvas-skeleton/pj_plot_widgets/sandbox/dock_drag_repro). It mirrors a typical ADS embedding (HiddenTitleBar + custom toolbar puppet-forwarding to the title bar) without any extra widgets. Set PJ_AUTO_DRAG=1 to fire a synthetic press / 8-step move / release sequence on dock A so the test runs without manual cursor input.

Tested under Ubuntu 26.04, Qt 6.8.3:

• QT_QPA_PLATFORM=xcb: drag works, identical UX to before this patch.
• QT_QPA_PLATFORM=wayland (native): drag works — preview tracks the cursor, drop indicators highlight correctly under the cursor's dock area, releases land on the indicated edge.

Repo / branch

This PR comes from PlotJuggler/Qt-Advanced-Docking-System:feature/wayland-fix, a single commit on top of upstream master 985ff74. We're carrying it as a submodule pin in our PJ4 application; happy to rebase / split / amend per review feedback.

[Wing-summer]

Glad to see you have some progress with QtAds on wayland. However, have you test the Demo on KDE wayland? On my CachyOS KDE wayland, the overlay still cannot be moved with following warning:

Warning: This plugin supports grabbing the mouse only for popup windows ((null):0, (null))

So I think you cannot get rid of Qt::Popup flag although it will make floating window in trouble. Or you can find another solution. Still X11 is OK in my env with your patch.

[Wing-summer]

If you can fully solve the wayland issue, I will close PR myself. I keep my PR open to remind other people who need this workaround. Thanks for your contribution.