[images-in-video] General feedback

[jakearchibald]

I really like this proposal. I think it's a step towards simplifying the media story on the web. It seems to me the ideal is:

• <video> supports media that has visual and/or audio components. It has a bunch of configuration, controls, and events.
• <audio> should just be <video> but without the visual part. If media is given to <audio> that doesn't have a visual component, it should error.
• <img> should just be <video> but without the audio part, without controls, and a stripped back API (although more could be added). If media is given to <img> that doesn't have a visual component, it should error.

Safari has made a good step towards this by supporting video formats in <img>.

Given that we have formats like animated AVIF, which are just repackagings of a video format, there's very little difference between "image" formats and "video" formats.

As for the .duration, it feels like the answer should be 0, unless 1-frame videos do something different.

[jakearchibald]

https://random-stuff.jakearchibald.com/media-combinations/ I created a little demo of various formats in different 'players'. Browsers generally already allow video/audio formats to work in both video/audio elements. Chrome lets AVIF into <video> but this seems like a bug. It won't play back the animated format.

[progers]

I also support this proposal. It's solving an important problem I've seen as well.

Chrome data indicates that animated svg images are pretty common, with ~2% of page loads having a smil-animated svg image, and I would guess css-animated svg images are similar. I think it would be useful to try to specify that svg images are supported, and treat them as having an indefinite duration. A potential additional benefit of this would be the ability to more reliably test animated svg images. Chromium's implementation currently has code that determines if an svg image contains smil or css animations. We would probably need to consider some additional edge-cases like base64 animated gifs and animated content in <foreignObject>, but I I think this is doable.

Question about user experience for animated gifs: the duration of a gif image may not be known, as frames can just keep streaming in. Would the duration be infinite as the gif loads, and then become defined once the gif has fully loaded, with appropriate updates to the video ui?

[noamr]

Question about user experience for animated gifs: the duration of a gif image may not be known, as frames can just keep streaming in. Would the duration be infinite as the gif loads, and then become defined once the gif has fully loaded, with appropriate updates to the video ui?

One thing that would be interesting to find out early on is whether very long GIFs (and other formats) can be downloaded from the middle with a byte-range request. e.g. what happens when you have a 10-minute GIF and seek it to the middle. I am not entirely sure that all the animated image formats support this but I'm not the expert.

It might be the case that we would have to download the entire GIF even when just a bit of it in the middle is played...

(I love the idea of the proposals but we should make sure that animated images can be accessed in the client/server sense in this way)