Interface GLMediaPlayer

All Superinterfaces:
TextureSequence

public interface GLMediaPlayer extends TextureSequence
GLMediaPlayer interface specifies a TextureSequence state machine using a multiplexed audio/video stream as it's source.

Audio maybe supported and played back internally or via an AudioSink implementation.

Audio and video streams can be selected or muted via initStream(Uri, int, int, int) using the appropriate stream id's.

Camera input can be selected using the CameraInputScheme Uri.

StreamWorker Decoding Thread

Most of the stream processing is performed on the decoding thread, a.k.a. StreamWorker:

StreamWorker generates it's own GLContext, shared with the one passed to initGL(GL). The shared GLContext allows the decoding thread to push the video frame data directly into the designated TextureSequence.TextureFrame, later returned via getNextTexture(GL) and used by the user.

StreamWorker Error Handling

Caught exceptions on StreamWorker are delivered as GLMediaPlayer.StreamExceptions, which either degrades the GLMediaPlayer.State to GLMediaPlayer.State.Uninitialized or GLMediaPlayer.State.Paused.

An occurring GLMediaPlayer.StreamException triggers a EVENT_CHANGE_ERR event, which can be listened to via GLMediaPlayer.GLMediaEventListener.attributesChanged(GLMediaPlayer, int, long).

An occurred GLMediaPlayer.StreamException can be read via getStreamException().

GLMediaPlayer Lifecycle

Action GLMediaPlayer.State Before GLMediaPlayer.State After Event
initStream(Uri, int, int, int) Uninitialized Initialized1, Uninitialized EVENT_CHANGE_INIT or ( EVENT_CHANGE_ERR + EVENT_CHANGE_UNINIT )
initGL(GL) Initialized Paused, , Uninitialized EVENT_CHANGE_PAUSE or ( EVENT_CHANGE_ERR + EVENT_CHANGE_UNINIT )
play() Paused Playing EVENT_CHANGE_PLAY
pause(boolean) Playing Paused EVENT_CHANGE_PAUSE
seek(int) Paused, Playing Paused, Playing none
getNextTexture(GL) Paused, Playing Paused, Playing none
getLastTexture() Paused, Playing Paused, Playing none
END_OF_STREAM Playing Paused EVENT_CHANGE_EOS + EVENT_CHANGE_PAUSE
GLMediaPlayer.StreamException ANY Paused, Uninitialized EVENT_CHANGE_ERR + ( EVENT_CHANGE_PAUSE or EVENT_CHANGE_UNINIT )
destroy(GL) ANY Uninitialized EVENT_CHANGE_UNINIT

Audio and video Stream IDs

value request get
STREAM_ID_NONE mute not available
STREAM_ID_AUTO auto unspecified
≥0 specific stream specific stream

Current implementations (check each API doc link for details):

  • NullGLMediaPlayer
  • OMXGLMediaPlayer
  • FFMPEGMediaPlayer
  • AndroidGLMediaPlayerAPI14

Implementations of this interface must implement:

    public static final boolean isAvailable();
 
to be properly considered by GLMediaPlayerFactory.create(ClassLoader, String) and GLMediaPlayerFactory.createDefault().

Timestamp Accuracy

Timestamp type and value range has been chosen to suit embedded CPUs and characteristics of audio and video streaming. See TimeFrameI.

Audio and video synchronization

The class follows a passive A/V synchronization pattern. Audio is being untouched, while getNextTexture(GL) delivers a new video frame only, if its timestamp is less than MAXIMUM_VIDEO_ASYNC ahead of time. If its timestamp is more than MAXIMUM_VIDEO_ASYNC ahead of time, the previous frame is returned. If its timestamp is more than MAXIMUM_VIDEO_ASYNC after time, the frame is dropped and the next frame is being fetched.

https://en.wikipedia.org/wiki/Audio_to_video_synchronization

   d_av = v_pts - a_pts;
 

Recommendation of audio/video pts time lead/lag at production:

  • Overall: +40ms and -60ms audio ahead video / audio after video
  • Each stage: +5ms and -15ms. audio ahead video / audio after video

Recommendation of av pts time lead/lag at presentation:

  • TV: +15ms and -45ms. audio ahead video / audio after video.
  • Film: +22ms and -22ms. audio ahead video / audio after video.

Test Streams

<!--
Big Buck Bunny 24f 16:9
Big Buck Bunny320ph264aac 48000Hz 2 chanhttp://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4
Big Buck Bunny240ph264aac 48000Hz 2 chanhttp://archive.org/download/BigBuckBunny_328/BigBuckBunny_512kb.mp4
Big Buck Bunny720pmpeg4ac3 48000Hz 5.1 chanhttp://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_surround.avi
Big Buck Bunny720pmsmpeg4v2mp3 48000Hz 2 chanhttp://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.avi
Big Buck Bunny720ptheoravorbis 48000Hz 2 chanhttp://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.ogg
Big Buck Bunny1080pmpeg4ac3 48000Hz 5.1 chanhttp://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_surround.avi
WebM/Matroska (vp8/vorbis)
Big Buck Bunny Trailer640pvp8vorbis 44100Hz 1 chanhttp://video.webmfiles.org/big-buck-bunny_trailer.webm
Elephants Dream540pvp8vorbis 44100Hz 1 chanhttp://video.webmfiles.org/elephants-dream.webm
You Tube http/rtsp
Sintelhttp://www.youtube.com/watch?v=eRsGyueVLvQrtsp://v3.cache1.c.youtube.com/CiILENy73wIaGQn0LpXnygYbeRMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp
Audio/Video Sync
Five-minute-sync-test1080phttps://www.youtube.com/watch?v=szoOsG9137Urtsp://v7.cache8.c.youtube.com/CiILENy73wIaGQm133VvsA46sxMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp
Audio-Video-Sync-Test-Calibration-23.98fps-24fpshttps://www.youtube.com/watch?v=cGgf_dbDMsw
sound_in_sync_testhttps://www.youtube.com/watch?v=O-zIZkhXNLE
title url1 url2

Since 2.3.0 this interface uses Uri instead of URI.