From c76760bc1ca4027441420752ff8e954292f58c8e Mon Sep 17 00:00:00 2001
From: Zulleyy3 Zulleyy3 <20947639+Zulleyy3@users.noreply.github.com>
Date: Sun, 29 Oct 2023 02:45:06 +0100
Subject: [PATCH] UI: Fix transform of sources in groups

Group resize is not deferred until adjustments are completed by a
mouseReleaseEvent in certain cases, resulting in unexpected movement of
sources that are part of a group as they are resized or stretched
beyond group bounds.

The bug can be described/reproduced as follow:

1. The user selects within a group in the sources dock
2. The user moves their mouse over the selected source such that the
   cursor would change.
   - This results in a cursor update and a call to GetStrechHandle()
   - as the source is part of a group, stretchGroup will be set to the
    group of the source

3. The user clicks on the canvas without touching a stretch/rotation
   handle
   - A mouseReleaseEvent is fired
   - obs_sceneitem_defer_group_resize_end is called on the group as
     stretchGroup is still defined from earlier
   - the defer_group_resize member of the group will become negative
   - The deferal check in resize_group (obs-scene.c) will always pass
     now
4. When scaling or rotating the source close to the group bounds, the
   group bounds will now dynamically update causing the source in to fly
   off the canvas.

Resolves: https://github.com/obsproject/obs-studio/issues/9754
---
 UI/window-basic-preview.cpp | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/UI/window-basic-preview.cpp b/UI/window-basic-preview.cpp
index c179815d0..83f998d55 100644
--- a/UI/window-basic-preview.cpp
+++ b/UI/window-basic-preview.cpp
@@ -520,6 +520,8 @@ void OBSBasicPreview::GetStretchHandleData(const vec2 &pos, bool ignoreGroup)
 							 &invGroupTransform);
 			matrix4_inv(&invGroupTransform, &invGroupTransform);
 			obs_sceneitem_defer_group_resize_begin(stretchGroup);
+		} else {
+			stretchGroup = nullptr;
 		}
 	}
 }