Phase 1 of the plugin manager provides the the ability to toggle off/on
plugins loading when OBS starts. Additionally, it implements loading of
a manifest file for plugins that allows plugin authors to provide more
detailed information about the plugin including authors, support site,
name, description.
In order to accomplish this, this change updates libobs to provide
more detailed tracking of modules- specifically tracking both enabled
and disabled modules, alone with a module load state which indicates
why a module is not loaded. Additionally, changes were made to establish
a links between a module and any features (inputs, outputs, encoders,
and services) it provides (and thus the ability to determine why a
feature might not be enabled). Along with these changes to modules,
this commit also provides an indicator and lookup for core modules which
can not be disabled by the plugin manager.
Finally, this change provides functions to properly load and retrieve
a standardized plugin metadata file.
Introduce support for delivering BPM (Broadcast
Performance Metrics) over SEI (for AVC/H.264 and
HEVC/H.265) and OBU (for AV1) unregistered messages.
Metrics being sent are the session frame counters,
per-rendition frame counters, and RFC3339-based
timestamping information to support end-to-end
latency measurement.
SEI/OBU messages are generated and sent with each IDR
frame, and the frame counters are diff-based, meaning
the counts reflect the diff between IDRs, not the running
totals.
BPM documentation is available at [1].
BPM relies on the recently introduced encoder packet timing
support and the packet callback mechanism.
BPM injection is enabled for an output by registering
the `bpm_inject()` callback via `obs_output_add_packet_callback()`
function. The callback must be unregistered using
`obs_output_remove_packet_callback()` and `bpm_destroy()`
must be used by the caller to release the BPM structures.
It is important to measure the number of frames successfully
encoded by the obs_encoder_t instances, particularly for
renditions where the encoded frame rate differs from the
canvas frame rate. The encoded_frames counter and
`obs_encoder_get_encoded_frames()` API is introduced
to measure and report this in the encoded rendition
metrics message.
[1] https://d50yg09cghihd.cloudfront.net/other/20240718-MultitrackVideoIntegrationGuide.pdf
Introduce support for the `encoder_packet_time` struct
to capture timing information for each frame, starting
from the composition of each frame, through the encoder,
to the queueing of the frame data to each output_t.
Timestamps for each of the following events are based on
`os_gettime_ns()`:
CTS: Composition time stamp (in the encoder render threads)
FER: Frame encode request
FERC: Frame encoder request completely
PIR: Packet interleave request (`send_interleaved()`)
Frame times are forwarded through encoder callbacks in the
context that runs on the relevant encoder thread, ensuring
no race conditions with accessing per encoder array happen.
All per-output processing happens on data that is owned by
the output.
Co-authored-by: Ruwen Hahn <haruwenz@twitch.tv>
Adds a new encoder cap which tells libobs that rather than scaling
video frames in software, the encoder is capable of scaling them via
its own (presumably more efficient) means.
An encoder may implement this cap by comparing the VOI of its assigned
`video_t` and the results of `obs_encoder_get_width/height()`. If the
width/height values differ, then the encoder is being asked by libobs
to self-scale, and the resolution in VOI will be the raw frame size,
with the `...get_width/height()` being the intended output resolution
of the encoder.
It is important to note that GPU rescaling mode will take priority
over self-scaling. If GPU rescaling is enabled, the encoder will never
be asked to self-scale.
This is useful for discrete hardware encoders, where they might have
fixed-function video scaling logic that is highly efficient and fast.
Additionally, this feature allows a hardware device which is encoding
a full ABR ladder of tracks to be smart and only copy a video frame
from GPU -> Host -> Device once for the entire ladder, rather than
once for every track.
This logic would previously have written any changed scale resolution
set by the encoder in the `.get_video_info` callback back to the
encoder, however this functionality was "broken" by
20d8779d30. In reality, this would have
never worked with texture encoders or with GPU rescaling enabled, and
probably would have had odd side effects for CPU rescaling, too. It's
best just to remove this functionality.
Fixes a crash when the following steps occur:
- Encoder group created and then started with encoders, only the group
owning encoder refs
- Encoder group destroy called: group has `destroy_on_stop` set
without actual destroy
- Outputs holding encoders from stopping are stopped
- `remove_connection()` function destroys encoder group, releasing
encoders and causing the currently processed encoder to be destroyed
early
- parent scopes try to access destroyed encoder pointer and crash
This change moves some logic around to improve the release/destruct
order of `obs_encoder_stop()` to fix the race above.
Note: Cases of encoder errors will not destruct the group if it has
`destroy_on_stop` set, as the encoder is also not destroyed if it also
is set to destroy on stop. This part of the code should be revisited
at a later date and fixed up to prevent memory leaks.
Modifies the encoder group API added previously to better follow the
existing libobs API naming paradigms. This also produces much more
readable code, and allows a few small benefits like only needing to
hold a reference to the encoder group, instead of every encoder.
When using a PTS divisor, OBS would still increment the PTS by only the
original `fps_den` value, not considering that PTS values should be
multiplied by the divisor.
For example, `60/1` increases like `0,1,2,3`. `60000/1001` increases
like `0,1001,2002,3003`.
Without this fix, `60/1` main OBS framerate with a divisor of `2`
produces `0,1,2,3`, while the correct pattern would be `0,2,4,6`
During encoder shutdown `maybe_clear_encoder_core_video_mix` is
called to clear created mixes that are no longer needed; at that
time `obs_encoder_set_video` rejects changes to `media` since
the encoder is still active, so we bypass those checks
This is an issue e.g. when a rtmp stream disconnects (and thus
all encoders are cleared) and subsequently reconnects
When #5169 implemented deferred encoder updates, it did not account for
the case when the encoder hadn't started yet. This means the encoder would
start and then immediately call update with the same settings it was
started with, which in the case of some hardware encoders would trigger
a reconfiguration request to the driver.
Reset frame_rate_divisor_counter to 0 on encoder shutdown.
After starting and stopping a GPU encoder session, obs_encoder_shutdown
would set frame_rate_divisor_counter to 1. When the next GPU encoder
session was started, in libobs/obs-video-gpu-encode.c, gpu_encode_thread
would set skip to this value (1), and increment
frame_rate_divisor_counter to 2. This causes the next check to fail, as
frame_rate_divisor is 1 by default (2 == 1 is false), so
frame_rate_divisor_counter retains its value. Since skip is non-zero,
the next check, if(skip), passes, and skip to the next loop iteration.
This will continue forever, because frame_rate_divisor_counter will
continue to increment, so it will never hold the same value as
frame_rate_divisor. This means that send_off_encoder_packet is never
called, so the muxer never receives encoded packets.
To the end-user, this manifests as their second encoder session being
impossible to stop. They then have to force quit OBS and the
obs-ffmpeg-mux process. This change prevents that from occurring and
allows multiple back-to-back GPU encoder sessions to be completed.
Allows rescaling resolution for GPU encoders and allows moving
rescaling for CPU encoders from CPU to GPU
Rescaling is implemented via core video mixes; encoders create
their own core mix with matching width/height/format/colorspace/
range when gpu scaling is enabled and no matching core video
mix exists
It was possible to set the encoder to scale the video to the same size
as the output, resulting in unnecessary passes through swscale and
skipping hardware encoders. This was a common user error in Advanced
Output mode where they would tick the rescale box but leave it at the
same resolution as their main output.
There is currently no way to clear a video_t or audio_t object from an
encoder once applied. `audio_t`/`video_t` objects can be destructed at
any time, and it is dangerous to prevent these object references from
even being cleared.
This does not fix the issue where destroying an audio/video object does
not clear the reference from all subscribed encoders.
Protect `obs_encoder_set_video()` and `obs_encoder_set_audio()` from
being used if the encoder is active. Changing these values while active
is undefined behavior.
Split render_texture and derived fields in obs_core_video into new
obs_core_video_mix struct. Add new APIs to add additional obs_view to the render loop, each with a separate render_texture / obs_core_video_mix.
With this, you can now cast normal obs objects (services, outputs,
sources, encoders) to an obs_object_t, and then use obs_object_*
functions to get references, release references, and similar for weak
object references as well. This allows the ability for the frontend to
use an object of any of those types interchangeably in certain
situations without having to handle each specific type individually.
This is useful because the properties view in particular doesn't care
what type of object it uses, it just needs to be able to hold weak
references to abstract OBS objects.
