Encoding skipping frames, incorrect PTS/DTS [FFMPEG]

[bentotom]

I've been trying to get an encode to work on FFMPEG, either though capture card or file.
Haven't figured out what wrong with my settings or if the plugin just doesen't play nice yet.

Almost immediately, the encode starts out with a PST/DTS desync and then the encode begins to skip frames.
CPU is at ~45%, and I can generally hit 115fps without any issues so I don't think its a system speed thing.

Here's the general command I'm using, although over the past 2 days I've used almost every permutation of pts/dts/vsync/startatzero/genpts/fps combinations.

-f dshow -rtbufsize 2048M -framerate 60 -s 1920x1080 -i video='AVerMedia HD Capture GC573 1':audio='AES (RME ADI-2 Pro)' -c:a libopus -ar 48000 -b:a 160k -c:v libsvt_vp9 -pix_fmt yuv420p -s 1920x1080 -r 60000/1000 -preset 9 -rc 2 -g 128 -b:v 15M -t 45 -f webm E:\VIDEO-TESTS\output\outputtestvp9-120fps.mkv

Logs included, maybe someone knows what I'm missing.

encodestats.log
ffmpeg-20200606-024045.log

[bentotom]

Also worth noting that the resulting file is generally playable but stutters on what I believe is the keyframe.
Adjusting the gop/kf value changes the tempo of the stutter

[tianjunwork]

Hi @guojiansheng0925 , could you take a look? Maybe start from transcode with the same configs first. Thank you.

[guojiansheng0925]

Hi @tianjunwork , it works well with local yuv files. I just got a camera and I will do more experiment tomorrow.

[bentotom]

@guojiansheng0925 Any idea why this might be happening?
I've tried this config with both files and capture and havent been able to generate a vp9 file without stutter on the keyframes. or a DTS mismatch

[guojiansheng0925]

Hi @bentotom , I am sorry I've no idea but I found a strange thing in ffmpeg-20200606-024045.log.
The version of FFmpeg is shown in the log:

ffmpeg version N-98035-gda5634c9ef-g801c8a961a+2 Copyright (c) 2000-2020 the FFmpeg developers

I think we should check the patch firstly. Are you working on the master branch and build this FFmpeg at the following commit?

801c8a961a504f014439c4312ecc3d66f03aea93 avcodec/ratecontrol: fix the integer overflow after long time run

Then how do you add the patch to FFmpeg?
Use git am? master-0001-Add-ability-for-ffmpeg-to-run-svt-vp9.patch?
Any warning or errors when you applying the patch?
If we apply the patch correctly, it will generate a new commit. And the version won't be related with the former commit id.

Secondly, if local input file can not work well in your environment, shall we try to simplify the issue? Could you please try the following command and check if the output video have stutter?
ffmpeg -s 720x1280 -i test.yuv -c:v libsvt_vp9 test.mkv

Thank you.

[1480c1]

ffmpeg version N-98035-gda5634c9ef-g801c8a961a+2 Copyright (c) 2000-2020 the FFmpeg developers

That +2 at the end means that he has 2 extra patches on top of the base commit he pulled from master

[1480c1]

It's set by

        _patches=$(git rev-list origin/master.. --count)
        [[ $_patches -gt 0 ]] &&
            do_addOption "--extra-version=g$(git rev-parse --short origin/master)+$_patches"

[bentotom]

I removed -g from the commands and it seemed to do the trick.
SVT-VP9 is defaulting to a gop size of 12 and everything seems smooth enough.
This might be worth looking into in the future though

[tianjunwork]

If gop size is not specified, IntraPeriod default is -2 which means auto mode. Encoder will decide when to add a key frame.
Hi @guojiansheng0925, did you see any playback issue with -g 128?

[bentotom]

Return shows, this.
Overall playback looks smooth. file size is slightly larger, but to be expected

SVT [config]: SourceWidth / SourceHeight                                        : 1920 / 1080

SVT [config]: Fps_Numerator / Fps_Denominator / Gop Size                : 120 / 1 / 12

[God-damnit-all]

If gop size is not specified, IntraPeriod default is -2 which means auto mode. Encoder will decide when to add a key frame.
Hi @guojiansheng0925, did you see any playback issue with -g 128?

I came here to look into this issue and -g 128 seems to solve it for me.

It was defaulting to 32 before.

[bentotom]

Tried -g 128 still seeing a hang/stutter.
Auto @ 12 seems to fix it for me for now.

Happy to do more tests or try different options.
Otherwise, feel free to close.

[God-damnit-all]

Hmm, doing -g 12 makes it stutter for me, though. How strange.

Edit: To clarify, leaving -g out like bentotom does, does NOT work for me.

[bentotom]

yeah, this might be an actual bug withe either the packager or the way SVT is handling keyframes.
A quick check of the frame sizes didn't show anything out of the ordinary, Keyframes are larger, as expected, but distance and frame durations were consistent.

And as I said, leaving -g out eliminates stutter for me

[tianjunwork]

Is that ok to attach a small clip of mkv with shutter issue?

[guojiansheng0925]

Hi @bentotom , Could you please check your svt-vp9 version?
On which commit id? Do you build it on a Windows System by MSYS?

[bentotom]

@tianjunwork
Here's a folder with two videos showing that issue. Videos encode fine with the below script without -g
https://drive.google.com/drive/folders/1z5gFNqQwJ29N9EO1cNPj7xbqr-VYDdY7?usp=sharing

-i ....MOV -an -s 1920x1080 -r 60 -c:v libsvt_vp9 -pix_fmt yuv420p -preset 9 -rc 2 -level 6 -g 128 -b:v 15M -t 90 .....mkv

[bentotom]

@guojiansheng0925
Commit ID e482c0a
Yes, built on a Windows system with MSYS

[tianjunwork]

I used VQA that can parse mkv container to read files attached by @bentotom.
With svt-vp9-g128.mkv, the Timecode for each SimpleBlock is increasing by 16(60fps) for each displayed frame. There is no issue with the timestamp in the container.
I also used VLC to playback the file. Enabled logging by tools -> messages -> verbosity: debug. Though there is warning saying frame is too late to be displayed. No frame is dropped. The playback has a few choppy moments, but overall is fine.
Assume your input source doesn't have choppy issue. I wonder if it is related to the decoding capability on different platform?

[tianjunwork]

Okay. The Timecode between cross each Cluster has a jump and is not continues.
1st cluster: Timecode 733, last SimpleBlock Timecode: 2032.
2nd cluster: Timecode 2867, last SimpleBlock Timecode: 2032.
3rd cluster: Timecode 5000 ...
733+2032 has gap with 2867. And the choppy happens just around crossing clusters.

BTW, frame is too late to be displayed is irrelevant. if I choose to slower the playback speed using vlc. This warning doesn't show up anymore. Seems decoder can't keep up with 60 fps. But it still shows choppy.

@guojiansheng0925 , could you take a further look at the timestamp crosses clusters?