FFmpeg lib + UDP + Qt issues

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

FFmpeg lib + UDP + Qt issues

First of all, thank you to this amazing community of developers!  I have received a tremendous amount of help both directly and indirectly thanks to you all.

I am in need of some help regarding reading encoded video udp packets and decoding them into a Qt display.  I have managed to read the video packets and display them in Qt but video playback 'stutters'.  Checking the FPS of frames transmitted via the FFMpeg library to Qt, I get ~30 frames per second, so I'm led to believe that FFMpeg itself is sending me the same frames multiple times a second.  Here are the steps I took:

    1. Broadcast an H.264 encoded video stream via VLC to udp:/

    2. Use FFMpeg lib to decode the stream.  Here is my function (apologies in advance, if there is a better way to display code, please let me know!):

bool QVideoDecoder::decodeSeekFrame(int after)
bool done=false;
// Read a frame
if(av_read_frame(pFormatCtx, &packet)<0)
return false;                             // Frame read failed (e.g. end of stream)

int frameFinished;

// Did we get a video frame?
ffmpeg::AVRational millisecondbase = {1, 1000};
int f = packet.dts;
int t = ffmpeg::av_rescale_q(packet.dts,pFormatCtx->streams[videoStream]->time_base,millisecondbase);
// If we decoded 2 frames in a row, the last times are okay
LastLastFrameTime = LastFrameTime;
LastLastFrameNumber = LastFrameNumber;

// Is this frame the desired frame?
if(after==-1 || LastFrameNumber>=after)
// Convert the image format (init the context the first time)
int w = pCodecCtx->width;
int h = pCodecCtx->height;
img_convert_ctx = ffmpeg::sws_getCachedContext(img_convert_ctx,w, h, pCodecCtx->pix_fmt, w, h, ffmpeg::PIX_FMT_RGB24, SWS_BICUBIC, NULL, NULL, NULL);

if(img_convert_ctx == NULL)
printf("Cannot initialize the conversion context!\n");
return false;
ffmpeg::sws_scale(img_convert_ctx, pFrame->data, pFrame->linesize, 0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize);

// Convert the frame to QImage

for(int y=0;y<h;y++)

// Set the time
DesiredFrameTime = ffmpeg::av_rescale_q(after,pFormatCtx->streams[videoStream]->time_base,millisecondbase);

done = true;

return done;

I have a periodic timer that requests the latest frame from FFMpeg. If I set the timer to a low rate (20 FPS), I get the video played back but at a slow rate.  If I set the timer to a high rate (60 FPS), I get the frames as soon as they are ready, but the video stutters.  I figured that I should request the frames back as fast as possible..  Any insight on how to avoid the stuttering is appreciated.

    3. Send the QImage back to Qt to be displayed -- I have tested the frames received by my Qt callback per second and can confirm that I am receiving ~ 30 FPS which leads me to believe the stuttering is caused by FFMpeg lib.

    4. I also wrote an interrupting callback to be called by FFMpeg if any of the blocking functions (i.e. av_read_frame) took too long:

Timer timeoutTimer;
static int decode_interrupt_cb(void)
if (timeoutTimer.getElapsedTimeInMilliSec() > 1000)
qDebug() << "Video Timed Out!";
return 1;
return 0;

Although I sometimes do receive the "Video Timed Out!" call, the av_read_frame function still blocks...

Any help would be greatly appreciated!