Hardware accelerated playback on PineTab 2 (RK3566)

Want to quickly document how I got my PineTab 2 to play 1080p videos smooth(-ish) with hardware acceleration.

TL;DR

PineTab2 can play 1080p 60fps perfectly smooth with hardware acceleration

Install `ffmpeg-v4l2-request-git` from the AUR

Play with `mpv --hwdec=drm --vo=gpu-next --wayland-disable-vsync=yes`

There'll be lag when MPV first shows control UI. But non after that.

If you are on a Wlroots based compositor, use `--vo=dmabuf-wayland` to reduce CPU usage

Enabling hardware acceleration

With the defaul DanctNIX image, it was quite easy. Simply install `mpv` and ` ffmpeg-v4l2-request-git` from the AUR. Then, pass pass `--hwdec=drm` to `mpv` and you're good to go. This will replace replace the system's `ffmpeg` with your custom build. If you run into ABI issues (random crashes), you can try building MPV from source.

For Ubuntu, you can simply look at Arch's PKGBUILD and replicate the steps.

Supported formats

According to Pine64's wiki, the mainline kernel supports the following formats:

Pine64 Wiki - Mainline Hardware Decoding

MPEG-2

H.264

VP8

**IMPORTANT**: As of writing, it DOES NOT support H.265/HEVC and VP9. Which YouTube uses by default. Thus, even though Chromium might pick up the fact that hardware acceleration is available, YouTube will still use software decoding. Also, this only supports up to 1080p even though the hardware is capable of higher resolutions.

Initial results

I did some quick tests with various videos I grabbed from YouTube and transcoded to H.264 using the `slower` preset. Then played them back with `mpv --hwdec=drm --vo=gpu-next` on KDE Plasma. The results are as follows:

Without hardware acceleration:

720p 24fps: No dropped frames

720p 60fps: ~30 dropped frames per second (half the frames missing!!)

1080p 60fps: Don't even bother

With hardware acceleration:

720p 24fps: No dropped frames

720p 60fps: ~1 dropped frame per second

1080p 25fps: No dropped frames

1080p 60fps: Running at about 45FPS in average.

Finally, seems like decoding H.264 is slightly more efficient than VP8. Stick to H.264 if you can.

Performance debugging

I'm curious about how the frame drops happened. Seems the hardware decoder is cabible. Something on the software side is messing up. Maybe some setting up can resolve it. I've noticed lag in KWin when I started using the PineTab 2. Maybe it's a compositor issue? Or is it actually because of copying the frames?

So I tried the `dmabuf-wayland` backend but MPV reported the compositor does not support passthrough for NV12. This forced me to use another compositor (Weston). It resulted in 20% less dropped frames. But at the cost of frame tearing when frames are dropped. (Weston is buggy on the PineTab 2, the terminal failed to render properly. I had to type commands blindly and redirect to see the output.)

Tried `--vo=gpu` but it resulted in 10% more dropped frames. Definitely stick to `--vo=gpu-next`.

Output scaling doesn't seem to affect performance.

My PineTab 2 playing a 1080p video. Dropped frames are shown in the bottom left corner.

Switching to Sway

I decided I don't want to use KDE. It's touch friendly. But I don't see myself using this "tablet" without a keyboard. A large in part due to planing on mostly coding on it. So I switched to Sway. That had an surprising effect on performance. I'm now getting almost no dropped frames on 1080p 60fps videos using `gpu-next` and fullscreened. Playback is perfectly smooth. Droppig only 600 frames in a 4:30 video.

The `dmabuf-wayland` is also qutie good. But more dropped frames. Which is very unexpected.

Perfect playback!

I set `GALLIUM_HUD="fps,cpu"` and discovered during 720p playback, the FPS is stadity right below 60. Instinctly I suspect this is a vsync issue. So I tried `--wayland-disable-vsync=yes` and it worked! No more dropped frames, not even in 1080p 60fps videos. The same is true on KDE. I suspect there's a bug in how the driver handles vsync.

720p 60fps: No dropped frames

1080p 60fps: No dropped frames

Furthermore, with a perfect playback. I can now use `--vo=dmabuf-wayland` to get a nice CPU use reduction. Playing a 1080p 60fps video now only uses 80% out of 400% CPU.

Yay!

MPV on PineTab 2 playing a 720p 60fps video. No dropped frames. (1080p is same, but I don't want to just demonstrate the same thing twice.)