To solve the problem of too high I/O

[shaonvpiaoyaodesilian]

  In my experimental verification, I/O overhead accounts for approximately 70-80% of the total time. Given this significant I/O overhead, I've considered using FFmpeg to decode the stream into a raw YUV420 byte stream and pipe it into a Python script to reduce I/O overhead. However, this would cause memory overflows due to the mismatch between the model speed and FFmpeg parsing speed. 
 I'd like to apply this model to real-world scenarios. Could you offer some new insights into addressing I/O issues?

[vovasilenko]

@shaonvpiaoyaodesilian, Have you implemented the algorithm from the article to convert FP16 to INT16 to support different platforms?

[shaonvpiaoyaodesilian]

@shaonvpiaoyaodesilian, Have you implemented the algorithm from the article to convert FP16 to INT16 to support different platforms?

No, I did not perform such an operation. The article clearly shows that FP16 is faster than Int16 in terms of encoding speed. Why would such a speed-down operation be necessary?

[shaonvpiaoyaodesilian]

By using pipelining and GPU-based color conversion, most of the overhead in this code, except for encoding/decoding time, can be eliminated.

[LiaoYujia]

通过使用流水线和基于 GPU 的颜色转换，除了编码/解码时间之外，该代码中的大部分开销都可以消除。

By using pipelining and GPU-based color conversion, most of the overhead in this code, except for encoding/decoding time, can be eliminated.

I’d love to learn more about your implementation. Could you share how you stream decoded frames from FFmpeg into Python and integrate GPU-accelerated preprocessing? Any chance you could share a snippet or workflow outline?

It would be a huge help for applying this in real-world settings.

Thanks!

[shaonvpiaoyaodesilian]

通过使用流水线和基于 GPU 的颜色转换，除了编码/解码时间之外，该代码中的大部分开销都可以消除。

By using pipelining and GPU-based color conversion, most of the overhead in this code, except for encoding/decoding time, can be eliminated.

I’d love to learn more about your implementation. Could you share how you stream decoded frames from FFmpeg into Python and integrate GPU-accelerated preprocessing? Any chance you could share a snippet or workflow outline?

It would be a huge help for applying this in real-world settings.

Thanks!

1.You should implement the streamReader function in the utils package.
2.Set the video stream address in json
3.Some color transformations will use GPU matrices.
4.Remove unnecessary performance computations.

[vovasilenko]

@shaonvpiaoyaodesilian, Have you implemented the algorithm from the article to convert FP16 to INT16 to support different platforms?

No, I did not perform such an operation. The article clearly shows that FP16 is faster than Int16 in terms of encoding speed. Why would such a speed-down operation be necessary?

@shaonvpiaoyaodesilian, For example, if you use an RTX 2080 TI for encoding and an RTX 4080 TI SUPER (or any other GPU) for decoding, you'll experience multiple artifacts. Read the article about solving this problem (determinism via INT16 or a corrective secondary bitstream).

[QLaHPD]

I also delved in such Intergerization problem, truth is you're better buying a hardware that will still be working in 100 years (if you're using it to archival purposes like me), than using the INT method, because its really much slower, and a little bit worse in terms of quality, so you better use something like AV1 which is much better supported.