Certain programs can start the virtualcam filter, then they may choose
to call `Stop()` on the filter, call `SetFormat()` to change the
resolution, then call `Run()` again to start the filter again. The
Windows virtual camera filter did not account for this, thus if the
resolution was different, it had potential to cause a crash.
To fix this, store the last filter resolution, then check the resolution
every frame, and if it changes, reset the scaling information.
(Author note: This code is unclean. What we need to do with the virtual
camera filter is make it only create the thread on `Run()`, then join
the thread on `Stop()`. It's currently a bit complicated to make it do
that at the moment, so this code is a kind of an annoying stopgap for
now.)
The `cx`/`cy`/`interval` variables specifically specify the
OBS/placeholder resolution/interval. The resolution may not be the same
as the filter's resolution (when scaling is used).
Instead, prefix these variables with `obs_` to improve clarity.
Sending frames on initial pause seems to cause an odd crash on
subsequent frame calls.
(Note by author: I do not know why the crash happened because code
beyond OBS is a proprietary black box. I suspect it's just a bug in
WebRTC or something, but I can't know for sure. This is incredibly
frustrating. But at least this particular crash seems to be fixed.
...for now.)
Apparently the testing that was done was not sufficient enough or did
not check all test cases, so resolution/fps does not get set properly on
the virtualcam.
This causes the thread to only start when the IMediaFilter::Run/Pause
calls have been made, and stop whenever either the IMediaFilter::Stop
call has been made, or on destruction, whichever comes first.
This potentially will work around a suspected race condition that
appears to be in the WebRTC library where the filter's library will be
released while the filter is in the process of being destroyed, which
can take longer than usual if the join takes too long. Basically, fixes
a reported crash (that doesn't appear to technically be our fault) when
the filter is used with browsers when the virtualcam is deactivating in
web browsers.
This may have been the issue behind the reported firefox crashes, but
strangely, the thread should always be joinable here because it's
explicitly always created in the constructor and joined in the
destructor.
This updates the libdshowcapture submodule to the latest commit.
This also changes win-dshow CMakeList to ensure compatibility.
Full commit list:
- Add support of RGB24 format
- CMake: Add Win32 libs instead of pragma directives.
- Implement FindPin
- Simplify error handling
- Fix log level of DebugHR
- Use default constructor instead of empty function
- Use std::move instead of copy where appropriate
- Mark some strings as const
My previous commits would rescale the placeholder for output on every
frame which was not CPU efficient. Now we store a copy of the scaled
placeholder and update it any time the resolution / FPS / format
changes.
A new function GetOutputBufferSize was added to allow calculating the
output size when needed. The existing code assumed a 12-bit format if
the placeholder didn't load, which could cause grey / green output.
An optimization was made to only load and scale the placeholder once
the virtual output starts, as many applications load and unload the
filter multiple times during enumeration of devices.
Various comments were also added to help clarify how the filter works.
The scaler assumed the placeholder was the same size as the camera which
caused crashes if the user replaced the placeholder with a smaller
resolution image (or if the camera was potentially running at > 1080p).
This adds a separate scaler for the placeholder and uses the resolution
of the virtual camera instead of defaulting to 1080p.
Per MSDN: Do not call this function from a DLL that is linked to the static C
run-time library (CRT). The static CRT requires DLL_THREAD_ATTACH and
DLL_THREAD_DETATCH notifications to function properly.
The virtual camera adds the ability to use the output of OBS itself as a
camera that can be selected within other Windows applications. This is
very loosely based upon the catxfish virtual camera plugin design.
There is a shared memory queue, but instead of having 10-20 frames in
the queue, there are now only 3 frames in the queue to minimize latency
and reduce memory usage. The third frame is mostly to ensure that
writing does not occur on the same frame being read; the delay is merely
one frame at all times.
The frames of the shared memory queue are NV12 instead of YUYV, which
reduces the memory and data copied, as well as eliminate unnecessary
conversion from NV12. Some programs (such as chrome, which uses webrtc
to capture) do not support NV12 however, so an I420 conversion is
provided, which is far less expensive than YUYV. The CPU cost of NV12
-> I420 is negligible in comparison.
The virtual camera filter itself is based upon the output filter within
the libdshowcapture library, which was originally implemented for other
purposes. This is more ideal than the Microsoft example code because
for one, it's far less convoluted, two, allows us to be able to
customize the filter to our needs a bit more easily, and three, has much
better RAII. The Microsoft CBaseFilter/etc code comprised of about 30
source files, where as the output filter comprises of two or three
required source files which we already had, so it's a huge win to
compile time.
Scaling is avoided whenever possible to minimize CPU usage. When the
virtual camera is activated in OBS, the width, height, and frame
interval are saved, that way if the filter is activated, it will always
remember the last OBS resolution/interval that the virtual camera was
activated with, even if OBS is not active. If for some reason the
filter activates before OBS starts up, and OBS starts up with a
different resolution, it will use simple point scaling intermittently,
and then will remember the new scaling in the future. The scaler could
use some optimization. FFmpeg was not opted for because the FFmpeg DLLs
would have to be provided for both architectures, which would be about
30 megabytes in total, and would make writing the plugin much more
painful. Thus a simple point scaling algorithm is used, and scaling is
avoided whenever possible.
(If another willing participant wants to have a go at improving the
scaling then go for it. But otherwise, it avoids scaling whenever
possible anyway, so it's not a huge deal)
