q->async_fifo grows too large when force idr, and few data written

[YaboWei]

OS: CentOS Linux release 7.2.1511 (Core)
CPU: i7-6770HQ
MSDK: 2017
repository: branch intel_plugins-3.3.1

I'm trying to follow the gop struct of the input, but few data written to output seconds later.

step 1:
blew command works well:
./ffmpeg_g -i "http://192.168.86.26:9000/live/s1_720p.flv" -vcodec h264_qsv -vsync passthrough -b:v 2000k -g 500 -async_depth 4 -acodec copy -f flv -y ~/output.flv

step 2:
modiry ffmpeg.c to keep picture type AV_PICTURE_TYPE_I
--- a/ffmpeg.c
+++ b/ffmpeg.c
@@ -1236,6 +1236,8 @@ static void do_video_out(OutputFile *of,
in_picture->quality = enc->global_quality;
in_picture->pict_type = 0;
++ if (in_picture->key_frame)
++ in_picture->pict_type = AV_PICTURE_TYPE_I;

and force idr:
./ffmpeg_g -i "http://192.168.86.26:9000/live/s1_720p.flv" -vcodec h264_qsv -vsync passthrough -force_idr 1 -b:v 2000k -g 500 -async_depth 4 -acodec copy -f flv -y ~/output.flv
after ffmpeg start a few seconds， output file ~/output.flv grows very slow.

do some work:
1, print q->async_fifo size：

q->async_fifo grows double in step 2：520,1040,2080,4160... ...,66560,133120,266240
only when q->async_fifo has no space, qsvenc read data from q->async_fifo and output them.

2, why q->async_fifo grows very quickly?
modify code in libavutil/fifo.c:
--- a/libavutil/fifo.c
+++ b/libavutil/fifo.c
@@ -113,7 +113,7 @@ int av_fifo_grow(AVFifoBuffer f, unsigned int size)
size += av_fifo_size(f);
if (old_size < size)
-- return av_fifo_realloc2(f, FFMAX(size, 2old_size));
++ return av_fifo_realloc2(f, FFMIN(size, 2*old_size));
return 0;
}

q->async_fifo grows slowly, and in step 2, output file ~/output.flv grows normally.

3, if q->async_fifo has more than ASYNC_FIFO_ELEM_SIZE data, qsvenc read data from q->async_fifo
modify libavcode/qsvenc.c
@@ -1253,15 +1258,18 @@ int ff_qsv_encode(AVCodecContext *avctx, QSVEncContext *q,
if (ret < 0)
return ret;
-- if (!av_fifo_space(q->async_fifo) ||
++ if ((frame && ASYNC_FIFO_ELEM_SIZE < av_fifo_size(q->async_fifo)) ||
(!frame && av_fifo_size(q->async_fifo))) {
AVPacket new_pkt;
mfxBitstream *bs;
mfxSyncPoint *sync;

in step 2, output file ~/output.flv grows normally, and q->async_fifo keep it's size.

questions:
1, why q->async_fifo always keep growing when force idr enabled?
2, why qsvenc read data from q->async_fifo only when q->async_fifo has no space?

[lizhong1008]

@YaboWei , testing it with another mp4 file, can't reproduce your issue. Haven't found the async_fifo size continually growing and the coded bitstream can be output. Maybe I need to reproduce it with your clip? Could you attach the "s1_720p.flv" file?

[YaboWei]

My test file:
https://drive.google.com/open?id=1TRA46Qd-y-5oVT099HFk2z7XdwxQOXLI

[lizhong1008]

Add the code as below:
diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c
index 19ea703..3e31edd 100644
--- a/libavcodec/qsvenc.c
+++ b/libavcodec/qsvenc.c
@@ -1253,6 +1253,9 @@ int ff_qsv_encode(AVCodecContext *avctx, QSVEncContext *q,
if (ret < 0)
return ret;

• printf("fifo_space size is %d\n", av_fifo_space(q->async_fifo));
• printf("fifo size is %d\n", av_fifo_size(q->async_fifo));

diff --git a/ffmpeg.c b/ffmpeg.c
index 2217fae..a8d8aba 100644
--- a/ffmpeg.c
+++ b/ffmpeg.c
@@ -1236,6 +1236,8 @@ static void do_video_out(OutputFile *of,

     in_picture->quality = enc->global_quality;
     in_picture->pict_type = 0;

•    if (in_picture->key_frame)
  

•        in_picture->pict_type = AV_PICTURE_TYPE_I;
  

Then ./ffmpeg_g -i s1_720p.flv -vcodec h264_qsv -vsync passthrough -force_idr 0 or 1 -b:v 2000k -g 500 -async_depth 4 -acodec copy -f flv -y ~/output.flv > log.txt
Haven't seen any different output for the "fifo size" and "fifo space size" in the log.txt.
Is there any thing wrong?

[YaboWei]

this is my output file:
https://drive.google.com/file/d/1nZ7dV6OvBzQNJGeg61qMUqOXDgr1SVTw/view?usp=sharing

[lizhong1008]

@YaboWei , have you check the out put when force_idr is closed? See my output: https://drive.google.com/drive/folders/1P1Bvu7t98mVWZYSAe1nBo4q95Zxk68A-?usp=sharing , no matter force_idr is 0 or 1, the output of fifo size are same.

[YaboWei]

Recently，I use MediaServerStudioProfessionalEvaluation2017R3 do the same test, the problem is still in.