Current code assumes that the overflow texture and the render target
always share the same color format including transfer function.
This assumption is incorrect however as OBS might use a 16-bit floating
point texture with a linear gamma transfer function when a high-bitrate
format is selected. On top of that OBS Studio wants rendering code to
ensure that linear color blending is used throughout the application.
The DrawOverflow function had not been updated to reflect this. This
change brings it in line with other functions involved in preview
rendering, which enable automatic sRGB conversion for the framebuffer
and the texture used in the fragment shader.
Certain draw calls were creating/destroying vertex buffers. Every call.
That's kind of not a great thing to do, so instead use the new
gs_draw_quadf() function to optimize rendering and reduce the need for
swapping vertex buffers.
Also uses a shader for DrawStripedLine so it does not have to split it
up into separate draw calls.
If the crop values combined are larger than the width or height of the
source, an integer overflow will occur.
This fix converts the width/height values to int, and then clamps
any negative values to 0.