Search Support

Avoid support scams. We will never ask you to call or text a phone number or share personal information. Please report suspicious activity using the “Report Abuse” option.

Learn More

How to efficiently display VideoFrames

  • Akukho zimpendulo
  • 0 zinale nkinga
  • 3 views
more options

Hi there,

apologies if this is not the right forum for a developer question, but Bugzilla didn't fell quite right in my case.

I'm developing a video streaming application that has to work with a custom streaming protocol. I already have an implementation using VideoDecoder and Insertable Streams that works in Chrome. Encoded VPX frames are being efficiently decoded and piped into an HTMLVideoElement via the MediaStreamTrack API. My understanding is that Firefox will soon have support for VideoDecoder (you can already enable it as an experimental feature and it works fine), but insertable streams is something that Firefox will not implement because it favors RTCRtpScriptTransform. Please correct me if I'm wrong.

Could you point me to a way how to efficiently get a VideoFrame that I got from a VideoDecoder into an HTMLVideoElement without a MediaStreamTrackGenerator (part of the insertable streams API which Firefox will not implement)? I've already tried:

  • drawing to a canvas (instead of video element)
  • wrapping the (still encoded) vpx frames in a WebM container and using it with a MediaSource SourceBuffer (with a video element)

Both attempts worked in Firefox, but performance was nowhere near the Insertable Streams way.

The VideoDecoders work very well in both Chrome and Firefox (with WebCodecs API enabled) according to my benchmarks. It's really just the matter of getting the decoded frame displayed where I can't find an efficient solution for Firefox.

Thanks a lot in advance!

If this is the wrong forum, could you please refer me to a more appropriate place?

Hi there, apologies if this is not the right forum for a developer question, but Bugzilla didn't fell quite right in my case. I'm developing a video streaming application that has to work with a custom streaming protocol. I already have an implementation using [https://developer.mozilla.org/en-US/docs/Web/API/VideoDecoder VideoDecoder] and [https://developer.mozilla.org/en-US/docs/Web/API/Insertable_Streams_for_MediaStreamTrack_API Insertable Streams] that works in Chrome. Encoded VPX frames are being efficiently decoded and piped into an HTMLVideoElement via the MediaStreamTrack API. My understanding is that Firefox will soon have support for VideoDecoder (you can already enable it as an experimental feature and it works fine), but insertable streams is something that Firefox will not implement because it favors [https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpScriptTransform RTCRtpScriptTransform]. Please correct me if I'm wrong. Could you point me to a way how to efficiently get a [https://developer.mozilla.org/en-US/docs/Web/API/VideoFrame VideoFrame] that I got from a VideoDecoder into an HTMLVideoElement without a MediaStreamTrackGenerator (part of the insertable streams API which Firefox will not implement)? I've already tried: * drawing to a canvas (instead of video element) * wrapping the (still encoded) vpx frames in a WebM container and using it with a MediaSource SourceBuffer (with a video element) Both attempts worked in Firefox, but performance was nowhere near the Insertable Streams way. The VideoDecoders work very well in both Chrome and Firefox (with WebCodecs API enabled) according to my benchmarks. It's really just the matter of getting the decoded frame displayed where I can't find an efficient solution for Firefox. Thanks a lot in advance! If this is the wrong forum, could you please refer me to a more appropriate place?

You must log in to your account to reply to posts. Please start a new question, if you do not have an account yet.