Video playback stutters more than necessary to keep video in sync with frame rate

Development Rendering
, ,
1

We have issues playing back video smoothly. We have identified that some screens will use a 59.94 Hz refresh rate while some use a 60 Hz so we have picked 59.94 as our target. We bake a movie that’s running at 29.97 Hz and expect it to be butter smooth on some screens and get a single frame glitch every 16 seconds to make up for the missing time. But, when this hitch happens it bounces back and forth for a while, which is an extremely unpleasant experience. If we encode the frame number into the movie and also print the rendering frames we can see that the playback alternates between delaying a frame and skipping a frame over a long period. Like there’s a floating point rounding error popping up like a ghost. The following are render frame numbers and movie frame numbers from an actual playback:

RF - MF

---------

920 - 454

921 - 454

922 - 455

923 - 455

924 - 456

925 - 456

926 - 457

927 - 458 (gained a frame)

928 - 458

929 - 458 (lost frame)

930 - 459

931 - 459

932 - 460

933 - 461 (gained a frame)

934 - 461

935 - 461 (lost a frame)

936 - 462

937 - 462

938 - 463

939 - 464 (gained a frame)

940 - 464

941 - 464 (lost a frame)

942 - 465

943 - 465

944 - 466

945 - 467 (gained a frame)

946 - 467

947 - 467 (lost a frame)

948 - 468

949 - 469 (gained a frame)

950 - 469

951 - 470

952 - 470

953 - 470 (lost a frame)

954 - 471

955 - 472 (gained a frame)

956 - 472

957 - 473

958 - 473

959 - 473 (lost a frame)

960 - 474

961 - 475 (gained a frame)

962 - 475

963 - 476

964 - 476

965 - 476 (lost a frame)

966 - 477

967 - 478 (gained a frame)

968 - 478

969 - 479

970 - 479

971 - 479 (lost a frame)

972 - 480

973 - 481

974 - 481

975 - 482

976 - 482

977 - 483

CPU or GPU isn’t bottlenecked. We are using Bink now but we had the same problem with the WmfMedia player along with some other performance issues. To me this looks like a bug but we don’t really have the bandwidth to dig into this. We have produced a clean reproduction case in vanilla UE 5.4.4.

2

Steps to Reproduce
This is reproduced in general by playing a movie rendered to 30 FPS on a TV with 59.94 FPS refresh rate or playing a movie rendered to 29.97 FPS and playing on a monitor with 60 FPS refresh rate.

I have added our reproduction case in vanilla UE 5.4.4. There are two UE projects. One suitable for 60 FPS screens and one for 59.94 FPS screens. Open one of the projects and play. Either you get no special hitches or you get a back-and-forth looking stuttering precisely every 16 second, that goes on for about a second. Which project gets the repro depends on the screen.

3

Hey, which media player are you using? From my experience the bink one works the best, I’ve had many issues with WMF / Electra players.

4

Bink uses the sound and a timer for frame progress, not the frame rate of the video (which varies all over the place on different machines, monitors, games, etc). So, when you have two clocks very close to each other they will drift in and out of phase slightly - as far I as I know, every codec works this same way.

You can completely take over frame advancing yourself by writing your own BinkWait and BinkShouldSkip functions that solely use the frame count to advance (you’ll have to handle when the game frame rate drops yourself, though). You’d have to rebuild the Bink plugin code, patch in your functions into the pBinkWait and pBinkShouldSkip function pointers, and then pipe through the game frame count somehow.

5

We’re using Bink. We used WMF before but had other issues as well.