Realtime motion interpolating 60fps playback in mpv

readme.md

Realtime motion interpolating playback in mpv

Edit 2016-09: SVP now has a free (but proprietary) version of their software, which includes a GPU port of the MVTools functions, making this much less resource intense: https://www.svp-team.com/wiki/SVP:Linux (AUR)

---

run this for SVP-like motion interpolation:

cd ~/.config/mpv # or ~/.mpv

wget https://gist.githubusercontent.com/phiresky/4bfcfbbd05b3c2ed8645/raw/motioninterpolation.vpy

echo 'I vf toggle format=yuv420p,vapoursynth=~~/motioninterpolation.vpy:4:4' >> input.conf

# to enable it by default
echo 'vf=format=yuv420p,vapoursynth=~~/motioninterpolation.vpy:4:4' >> mpv.conf

now press shift+i while mpv is playing a video to toggle beautiful motion vector interpolation of the video framerate to your display refresh rate.

You need mpv compiled with VapourSynth support. Not sure what distributions do this, in Arch Linux you can get mpv-git or mpv-vapoursynth (AUR) and vapoursynth-plugin-mvtools.

It should also be possible to make this work through SMPlayer.

Warning: This is CPU intensive. The settings work on a i5-2500k cpu. If it lags you will have to change some parameters, look for 2500k in the script. If your CPU is faster remove that block.

If your CPU is really fast, try removing the 'mode' parameter and changing BlockFPS to FlowFPS, this gives much better results.

See the reddit post for more information

Encoding video to a higher framerate

Run this:

vspipe --arg in_filename=input.mkv --arg display_fps=60 --y4m motioninterpolation.vpy -|ffmpeg -i - -crf 18 output.mkv

motioninterpolation.vpy

# vim: set ft=python:

# see the README at https://gist.github.com/phiresky/4bfcfbbd05b3c2ed8645
# source: https://github.com/mpv-player/mpv/issues/2149
# source: https://github.com/mpv-player/mpv/issues/566
# source: https://github.com/haasn/gentoo-conf/blob/nanodesu/home/nand/.mpv/filters/mvtools.vpy

import vapoursynth

core = vapoursynth.get_core()
# ref: http://avisynth.org.ru/mvtools/mvtools2.html#functions
# default is 400, less means interpolation will only happen when it will work well
ignore_threshold=140
# if n% of blocks change more than threshold then don't interpolate at all (default is 51%)
scene_change_percentage=15

dst_fps = display_fps
# Interpolating to fps higher than 60 is too CPU-expensive, smoothmotion can handle the rest.
# while (dst_fps > 60):
#    dst_fps /= 2

if "video_in" in globals():
    # realtime
    clip = video_in
    # Needed because clip FPS is missing
    src_fps_num = int(container_fps * 1e8)
    src_fps_den = int(1e8)
    clip = core.std.AssumeFPS(clip, fpsnum = src_fps_num, fpsden = src_fps_den)
else:
    # run with vspipe
    clip = core.ffms2.Source(source=in_filename)
    dst_fps=float(dst_fps)

# resolution in megapixels. 1080p ≈ 2MP, 720p ≈ 1MP
mpix = clip.width * clip.height / 1000000

# Skip interpolation for >1080p or 60 Hz content due to performance
if not (mpix > 2.5 or clip.fps_num/clip.fps_den > 59):
    analParams = {
        'overlap': 0,
        'search': 3,
        'truemotion': True,
        #'chrome': True,
        #'blksize':16,
        #'searchparam':1
    }
    blockParams = {
        'thscd1': ignore_threshold,
        'thscd2': int(scene_change_percentage*255/100),
        'mode': 3,
    }

    if mpix > 1.5:
        # can't handle these on Full HD with Intel i5-2500k
        # see the description of these parameters in http://avisynth.org.ru/mvtools/mvtools2.html#functions
        analParams['search'] = 0
        blockParams['mode'] = 0
        quality = 'low'
    else:
        quality = 'high'


    dst_fps_num = int(dst_fps * 1e4)
    dst_fps_den = int(1e4)
    print("Reflowing from {} fps to {} fps (quality={})".format(clip.fps_num/clip.fps_den,dst_fps_num/dst_fps_den,quality))
    
    sup  = core.mv.Super(clip, pel=2)
    bvec = core.mv.Analyse(sup, isb=True, **analParams)
    fvec = core.mv.Analyse(sup, isb=False, **analParams)
    clip = core.mv.BlockFPS(clip, sup, bvec, fvec, 
            num=dst_fps_num, den=dst_fps_den, 
            **blockParams)

clip.set_output()

@Cellane not sure. But you could add other tracks to the ffmpeg command (see here).

2 small things, Shift+I by default now toggles stats, there are numerous undefined shift+'s you could use.
Also this warning is printed, not an issue yet (given current development pace may be a long time till removed?
"Passing more than 1 argument to vf-toggle is deprecated!"

script is currently a no go with recent vapoursynth -
[vapoursynth] Script evaluation failed:
[vapoursynth] Python exception: No attribute with the name mv exists. Did you mistype a plugin namespace?
[vapoursynth]
[vapoursynth] Traceback (most recent call last):
[vapoursynth] File "src/cython/vapoursynth.pyx", line 1847, in vapoursynth.vpy_evaluateScript
[vapoursynth] File "/home/doug/.config/mpv/motioninterpolation.vpy", line 61, in
[vapoursynth] sup = core.mv.Super(clip, pel=2)
[vapoursynth] File "src/cython/vapoursynth.pyx", line 1416, in vapoursynth.Core.getattr
[vapoursynth] AttributeError: No attribute with the name mv exists. Did you mistype a plugin namespace?
[vapoursynth]
[vapoursynth] could not init VS
Disabling filter vapoursynth.00 because it has failed.

Ignore above, forgot to install a mvtools plugin ...

I actually need help settings this up correctly on windows 10. anytime I enabled or attempt to enable the filter it fails and If i try to enabled it again it locks up mpv and force closes it. I have mvtools plugin, and mvtools.vpy (never actually installed it though i dont know how) but honestly I don't know what to do with either of these or where to place them. I'm really new to the whole script thing with mpv.

I do have vapoursynth with mpv compiled already. I just really want to get this filter to work but know little to nothing about how to do this.

I believe similar issue to above

I cant get a Vapoursynth version of mpv on my ubuntu 18. same with spv. i tried to compile it, but it has alot of unresolved dependencies.

isnt there a way to start vapoursynth and feed into a file and load that file into a media player - that was the way id was done with avisynth a time ago, wasnt it?

thx

I have two issues with the script:
First: It shows me that "AttributeError: No attribute with the name mv exists. Did you mistype a plugin namespace?"
While I brew installed mvtools (I'm on macOS)
Second: When I try to first enable SVP it seems to work but another error is raised "AssumeFPS: invalid framerate specified" (I have SVP for 30 days)
If someone could help me it would be nice!

Sorry, can't really help you.. The attribute error means that mvtools is not installed in a way that mpv can find it. I don't know how to verify this on MacOS. Regarding SVP, maybe you're getting a conflict between the above script and SVP? Try removing / emptying your mpv config and seeing if that helps..

@Animenosekai I ran into the same issue on macOS. The trouble is that the MVTools plugin isn't being autoloaded by vapoursynth. This isn't related to SVP.

You can fix this by loading the mvtools plugin explicitly right after core is defined. So after the comments these should be the first lines of code:

...
import vapoursynth

core = vapoursynth.get_core()
core.std.LoadPlugin(path="/usr/local/lib/libmvtools.dylib")  # path where homebrew installs mvtools on macOS

This is explained in the caveats for the homebrew mvtools formula.

@phiresky to help out mac users, you might consider adding a commented out # core.std.LoadPlugin(....) line into your gist with a note explaining it :-)

hi dudes i dont know how to make this work T_T i aready have installed mpv python and vapoursynth on windows 10 but i don't have any clue how to make this work can someone help me plz ?

EDIT 2: Finally managed to downgrade to Vapoursynth R53, all working good.

---

EDIT 1: After re-installing vapoursynth through pip, and removing the format=yuv420p argument, I get this error(see following debug log)

[cplayer] first video frame after restart shown
[cplayer] audio ready
[cplayer] starting audio playback
[cplayer] playback restart complete @ 0.023000, audio=playing, video=playing
[ao/pulse] starting AO
[statusline] AV: 00:00:00 / 00:04:27 (0%) A-V: -0.000
[vo/gpu/x11] Disabling screensaver.
[cplayer] Set property: shared-script-properties -> 1
[cplayer] Set property: shared-script-properties -> 1
[statusline] AV: 00:00:02 / 00:04:27 (1%) A-V:  0.000
[user_filter_wrapper] Setting option 'file' = '~~/motioninterpolation.vpy' (flags = 0)
[user_filter_wrapper] Setting option 'buffered-frames' = '4' (flags = 0)
[user_filter_wrapper] Setting option 'concurrent-frames' = '4' (flags = 0)
[vapoursynth] using 4 concurrent requests.
[vapoursynth] Could not initialize VapourSynth scripting.
[user_filter_wrapper] Creating filter 'vapoursynth' failed.

It just does not initialize...

---

Vapoursynth R56 broke everything. It's not working now. Here is a debug. I cannot downgrade because of dependency hell(my fault) but I think this should be helpful anyways.

Passing more than 1 argument to vf-toggle is deprecated!
[vapoursynth] Script evaluation failed:
[vapoursynth] Python exception: module 'vapoursynth' has no attribute 'get_core'
[vapoursynth] 
[vapoursynth] Traceback (most recent call last):
[vapoursynth]   File "src/cython/vapoursynth.pyx", line 2759, in vapoursynth._vpy_evaluate
[vapoursynth]   File "src/cython/vapoursynth.pyx", line 2760, in vapoursynth._vpy_evaluate
[vapoursynth]   File "/home/az/.config/mpv/motioninterpolation.vpy", line 10, in <module>
[vapoursynth]     core = vapoursynth.get_core()
[vapoursynth] AttributeError: module 'vapoursynth' has no attribute 'get_core'
[vapoursynth] 
[vapoursynth] could not init VS
Disabling filter vapoursynth.00 because it has failed.

Using same motioninterpolation.vpy as above.

https://forum.videohelp.com/threads/403295-Installing-vapoursynth#post2632997

Ok. So I wanted to try to get r56 running again. I changed core = vapoursynth.get_core() to core = vapoursynth.core() but I think that is not enough and that more needs to be change. Perhaps a rewritten updated motion interpolation script. Not even r53 works anymore.
Here is the log(for r56). It might be helpful.

[vapoursynth] Script evaluation failed:
[vapoursynth] Python exception: 'vapoursynth._CoreProxy' object is not callable
[vapoursynth] 
[vapoursynth] Traceback (most recent call last):
[vapoursynth]   File "src\cython\vapoursynth.pyx", line 2832, in vapoursynth._vpy_evaluate
[vapoursynth]   File "src\cython\vapoursynth.pyx", line 2833, in vapoursynth._vpy_evaluate
[vapoursynth]   File "/home/az/.config/mpv/motioninterpolation.vpy", line 10, in <module>
[vapoursynth]     core = vapoursynth.core()
[vapoursynth] TypeError: 'vapoursynth._CoreProxy' object is not callable
[vapoursynth] 
[vapoursynth] could not init VS
Disabling filter vapoursynth.00 because it has failed.

It's not core = vapoursynth.core(), it's core = vapoursynth.core. No parentheses.

It's not core = vapoursynth.core(), it's core = vapoursynth.core. No parentheses.
🤦 Thank you. 🤦 It works again now. Wow...

Whats causing this issue

[cplayer] Run command: vf, flags=73, args=[operation="toggle", value="format=yuv420p,vapoursynth=/motioninterpolation.vpy:4:4"]
[cplayer] Passing more than 1 argument to vf-toggle is deprecated!
[user_filter_wrapper] Setting option 'fmt' = 'yuv420p' (flags = 0)
[user_filter_wrapper] Setting option 'file' = '/motioninterpolation.vpy' (flags = 0)
[user_filter_wrapper] Setting option 'buffered-frames' = '4' (flags = 0)
[user_filter_wrapper] Setting option 'concurrent-frames' = '4' (flags = 0)
[global] config path: 'motioninterpolation.vpy' -> '/home/triplek/.config/mpv/motioninterpolation.vpy'
[global] user path: '~~/motioninterpolation.vpy' -> '/home/triplek/.config/mpv/motioninterpolation.vpy'
[vapoursynth] using 4 concurrent requests.
[vapoursynth] Could not initialize VapourSynth scripting.
[user_filter_wrapper] Creating filter 'vapoursynth' failed.

Could not initialize VapourSynth scripting.

No python installed (/ correctly).

Thanks for the script. Here were the steps required for fedora 35.

1. sudo dnf install fftw-devel nasm fftw vapoursynth-devel vapoursynth-plugins vapoursynth-tools python3-vapoursynth
2. git clone https://github.com/dubhater/vapoursynth-mvtools
3. Then follow the instructions in the repo above under compilation, libmvtools will be built in the build dir inside the repo.
4. Then see the comment : https://gist.github.com/phiresky/4bfcfbbd05b3c2ed8645?permalink_comment_id=3310945#gistcomment-3310945 and add the path to the library

Install instructions for NixOS:

Instead of using pkgs.mpv in environment.systemPackages, use:

(pkgs.wrapMpv
  (pkgs.mpv-unwrapped.override { vapoursynthSupport = true; })
  {
    extraMakeWrapperArgs = [
      "--prefix" "LD_LIBRARY_PATH" ":" "${pkgs.vapoursynth-mvtools}/lib"
    ];
  })

Install the motioninterpolation.vpy as described and replace vapoursynth.get_core() with just vapoursynth.core. And add below (mvtools library won't be found otherwise):

import os
for d in os.environ["LD_LIBRARY_PATH"].split(":"):
    core.std.LoadAllPlugins(os.path.join(d, "vapoursynth"))

The correct way to install vapoursynth plugins on NixOS is to use vapoursynth.withPlugins.
For mpv you would do:

    mpv-unwrapped = super.mpv-unwrapped.override {
      vapoursynthSupport = true;
      vapoursynth = with self; vapoursynth.withPlugins
        [ vapoursynth-mvtools ffms ];
    };

As of 2024-02-01, on arch, is mpv-git or mpv-vapoursynth still required (instead of just mpv)? thanks!

I'm trying to use the encode command:
vspipe --arg in_filename=input.mkv --arg display_fps=60 --y4m motioninterpolation.vpy -|ffmpeg -i - -crf 18 output.mkv
It's returning the error:
[in#0 @ 0x62f666cc84c0] Error opening input: Invalid data found when processing input Error opening input file -. Error opening input files: Invalid data found when processing input
Thanks!