libobs: Add sanity checks to some obs_output_t functions

Since OBS outputs can have any combination of flags for encoded,
video, audio, and service, there are a number of cases where it would
be a good idea to validate that we're not allowing output changes that
risk undefined behavior given the supported flags.

This also does a mild amount of code cleanup, adding inline functions
for checking the flags mentioned above.
This commit is contained in:
Internal Server Error - Gitea: Git with a cup of tea
500 Internal Server Error

Gitea Version: 1.28.0+dev-477-g8b6ad49a5f