OBS-Studio and Hardware Encoding for Linux

There are a lot of different ways to do hardware based encoding with obs-studio, however, this particular article will discuss getting things to work with the libva api standard that some Intel and AMD gpu’s support.   Unfortunately, obs-studio doesn’t support this right out of the box, but since it is open source, several people have made the necessary files available.

First though why do you want Hardware Encoding of your videos, obs-studio by default comes with access to the x264 software based encoder that is used with ffmpeg.  While this does a good enough job, it is also very CPU intensive, and in my experience this is is pretty picky on CPU usage that can start to drop your frame’s per seconds down.  Dropped frames means stuttering or laggy video.  Which isn’t a good thing.

On my particular system using software encoding with OBS-Studio running would take between 18 to 25 percent cpu.  If it creeped up into the 20% range, it would  start to drop frames.   So I had to be judicial about what was being done with the scenes.   With Hardware encoding enabled, the system is consistently using between 5 to 8 percent CPU.  A huge difference that frees up the CPU to be used for other things by other applications.

If you have Ubuntu installed, you probably already have the necessary drivers needed to use LIBVA api for hardware encoding.   To check this you can run the command vainfo to get the necessary information if your system supports hardware encoding.

vainfo libva info: VA-API version
    0.38.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib64/dri/i965_drv_video.so libva info: Found init function
    __vaDriverInit_0_38 libva info: va_openDriver() returns 0 vainfo: VA-API version: 0.38 (libva 1.6.1) vainfo: Driver version: Intel i965 driver for Intel(R) Haswell Desktop -
    1.6.1 vainfo: Supported profile and entrypoints
          VAProfileMPEG2Simple            : VAEntrypointVLD
          VAProfileMPEG2Simple            : VAEntrypointEncSlice
          VAProfileMPEG2Main              : VAEntrypointVLD
          VAProfileMPEG2Main              : VAEntrypointEncSlice
          VAProfileH264ConstrainedBaseline: VAEntrypointVLD
          VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
          VAProfileH264Main               : VAEntrypointVLD
          VAProfileH264Main               : VAEntrypointEncSlice
          VAProfileH264High               : VAEntrypointVLD
          VAProfileH264High               : VAEntrypointEncSlice
          VAProfileH264MultiviewHigh      : VAEntrypointVLD
          VAProfileH264MultiviewHigh      : VAEntrypointEncSlice
          VAProfileH264StereoHigh         : VAEntrypointVLD
          VAProfileH264StereoHigh         : VAEntrypointEncSlice
          VAProfileVC1Simple              : VAEntrypointVLD
          VAProfileVC1Main                : VAEntrypointVLD
          VAProfileVC1Advanced            : VAEntrypointVLD
          VAProfileNone                   : VAEntrypointVideoProc
          VAProfileJPEGBaseline           : VAEntrypointVLD
          VAProfileH264MultiviewHigh      : VAEntrypointVLD
          VAProfileH264MultiviewHigh      : VAEntrypointEncSlice
          VAProfileH264StereoHigh         : VAEntrypointVLD
          VAProfileH264StereoHigh         : VAEntrypointEncSlice

If you get information like above, you are set to move to the next step.  If you don’t, then either your system doesn’t support libva, or you may need to install the necessary deb file (intel-libva-driver I believe).

Next we need to recompile OBS-Studio with the necessary support plugin.  Fortunately, Reboot from GitHub has create the necessary plugin for us.   I’ve forked his code, and updated it to work with 19.x or above.   You will need to clone the repository and check out the vaapi-h264 branch I have.

I’m not going to go into detail on how to compile OBS-Studio, that is documented on the Wiki.   But once you recompile, you should now have a new menu option in the Output section of the preferences.   It will still default to x264, but you should be able to select the VA-API encoder option.   I would recommend this for both Streaming and Recording.  You shouldn’t need to mess with changing any of the defaults.

Start your recordings, and you should not see your CPU spiking as you did in the past.  You can play with the settings if you want, but the defaults worked well for me.




This entry was posted in obs-studio, streaming, Uncategorized and tagged , . Bookmark the permalink.

2 Responses to OBS-Studio and Hardware Encoding for Linux

  1. Any idea for this error:

    info: Settings changed (outputs)
    info: ————————————————
    info: [VAAPI encoder]: settings:

    device_type : X11
    device: Intel i965 driver for Intel(R) CherryView – 1.8.3
    device_path: (null)
    fps_num: 30
    fps_den: 1
    width: 1280
    height: 720
    profile: main
    keyint: 2 (s)
    rc_type: CBR
    bitrate: 2500 (kbit)
    use_custom_cpb: no

    info: libfdk_aac encoder created
    info: libfdk_aac bitrate: 160, channels: 2
    info: ==== Recording Start ===============================================
    info: [ffmpeg muxer: ‘adv_file_output’] Writing file ‘/home/default/2018-03-22 03-31-50.flv’…
    error: [VAAPI encoder]: “vaEndPicture(q->display, q->context)”: invalid parameter
    error: [VAAPI encoder]: unable to encode frame
    error: Error encoding with encoder ‘streaming_h264’

    Changing the device path (in OBS) to “device_path: /dev/dri/card0” is no use either

    Currently I’m unable to output with the h264-vaapi build using x264 anyway, so might be related to it as well..

    Using Cherry trail x5-z8300. On Windows OBS works fine with qsv..

    • kingargyle says:

      What distribution of Linux are you using. The only version I’ve gotten it to work on consistently is Ubuntu 16.04. That is due to the VAAPI drivers become more strict and a bug in the obs studio plugin not setting a flag correctly. Unfortunately I haven’t had any time to try and get it to work with newer versions of the drive. The invalid parameter you are getting is a symptom of this bug. I did post some information in the PR on how it should be fixed, I just haven’t had time to fix it myself.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s