@UnstableApi
public interface VideoSink

A sink that consumes decoded video frames and images.

Summary

Nested types
@Documented
@Retention(value = RetentionPolicy.SOURCE)
@Target(value = TYPE_USE)
@UnstableApi
@IntDef(value = )
public annotation VideoSink.FirstFrameReleaseInstruction

The instruction provided when the stream changes for releasing the first frame.
@Documented
@Retention(value = RetentionPolicy.SOURCE)
@Target(value = TYPE_USE)
@IntDef(value = )
public annotation VideoSink.InputType

Specifies how the input frames are made available to the video sink.
public interface VideoSink.Listener

Listener for VideoSink events.

Handler for a video frame.
public final class VideoSink.VideoSinkException extends Exception

Thrown by VideoSink implementations.

Constants
default static final int

Input frames come from a Bitmap.
default static final int

Input frames come from a surface.
default static final int

Instructs to release the first frame as soon as possible.
default static final int

Instructs to release the first frame when the playback position reaches the stream start position.
default static final int

Instructs to release the first frame when rendering starts.

Public methods
abstract void

Allows the sink to release the first frame even if rendering is not started.
abstract void

Clears the set output surface info.
abstract void
flush(boolean resetPosition)

Flushes the video sink.
abstract Surface

Returns the input Surface where the video sink consumes input frames from.
abstract boolean
handleInputBitmap(
    Bitmap inputBitmap,
    TimestampIterator bufferTimestampIterator
)

Handles an input Bitmap.
abstract boolean
handleInputFrame(
    long bufferPresentationTimeUs,
    VideoSink.VideoFrameHandler videoFrameHandler
)

Handles a video input frame.
abstract boolean
initialize(Format sourceFormat)

Initializes the video sink.
abstract boolean

Returns whether all the data has been rendered to the output surface.
abstract boolean

Returns whether the video sink is initialized.
abstract boolean
isReady(boolean otherwiseReady)

Returns whether the video sink is able to immediately render media to its output surface from the current position.
abstract void
join(boolean renderNextFrameImmediately)

Joins the video sink to a new stream.
abstract void
onInputStreamChanged(
    @VideoSink.InputType int inputType,
    Format format,
    long startPositionUs,
    @VideoSink.FirstFrameReleaseInstruction int firstFrameReleaseInstruction,
    List<Effect> videoEffects
)

Informs the video sink that a new input stream will be queued with the given effects.
abstract void

Redraws the video effects immediately.
abstract void

Releases the sink.
abstract void
render(long positionUs, long elapsedRealtimeUs)

Incrementally renders processed video frames to the output surface.
abstract void
setBufferTimestampAdjustmentUs(long bufferTimestampAdjustmentUs)

Sets the buffer timestamp adjustment.
abstract void
setChangeFrameRateStrategy(
    @C.VideoChangeFrameRateStrategy int changeFrameRateStrategy
)

Changes the C.VideoChangeFrameRateStrategy used when calling setFrameRate.
abstract void
setListener(VideoSink.Listener listener, Executor executor)

Sets a Listener on this sink.
abstract void
setOutputSurfaceInfo(Surface outputSurface, Size outputResolution)

Sets the output surface info.
abstract void
setPlaybackSpeed(@FloatRange(from = 0, fromInclusive = false) float speed)

Sets the playback speed.
abstract void
setVideoEffects(List<Effect> videoEffects)

Sets video effects to apply immediately.
abstract void
setVideoFrameMetadataListener(
    VideoFrameMetadataListener videoFrameMetadataListener
)

Sets the VideoFrameMetadataListener.
abstract void

Signals the end of the current input stream.
abstract void

Signals the end of the last input stream.
abstract void

Starts rendering to the output surface.
abstract void

Stops rendering to the output surface.

Constants

INPUT_TYPE_BITMAP

default static final int INPUT_TYPE_BITMAP = 2

Input frames come from a Bitmap.

INPUT_TYPE_SURFACE

default static final int INPUT_TYPE_SURFACE = 1

Input frames come from a surface.

RELEASE_FIRST_FRAME_IMMEDIATELY

default static final int RELEASE_FIRST_FRAME_IMMEDIATELY = 0

Instructs to release the first frame as soon as possible.

RELEASE_FIRST_FRAME_WHEN_PREVIOUS_STREAM_PROCESSED

default static final int RELEASE_FIRST_FRAME_WHEN_PREVIOUS_STREAM_PROCESSED = 2

Instructs to release the first frame when the playback position reaches the stream start position.

RELEASE_FIRST_FRAME_WHEN_STARTED

default static final int RELEASE_FIRST_FRAME_WHEN_STARTED = 1

Instructs to release the first frame when rendering starts.

Public methods

allowReleaseFirstFrameBeforeStarted

abstract void allowReleaseFirstFrameBeforeStarted()

Allows the sink to release the first frame even if rendering is not started.

This is used to update the FirstFrameReleaseInstruction of the stream that is currently being processed.

clearOutputSurfaceInfo

abstract void clearOutputSurfaceInfo()

Clears the set output surface info.

flush

abstract void flush(boolean resetPosition)

Flushes the video sink.

After calling this method, any frames stored inside the video sink are discarded.

Parameters
boolean resetPosition

Whether to reset the current position.

getInputSurface

abstract Surface getInputSurface()

Returns the input Surface where the video sink consumes input frames from.

Must be called after the sink is initialized.

handleInputBitmap

abstract boolean handleInputBitmap(
    Bitmap inputBitmap,
    TimestampIterator bufferTimestampIterator
)

Handles an input Bitmap.

Must be called after the corresponding stream is signaled.

Parameters
Bitmap inputBitmap

The Bitmap to queue to the video sink.
TimestampIterator bufferTimestampIterator

The buffer presentation times within the current stream that the bitmap should be shown at. The timestamps should be monotonically increasing.
Returns
boolean

Whether the bitmap was queued successfully. If false, the caller can try again later.

handleInputFrame

abstract boolean handleInputFrame(
    long bufferPresentationTimeUs,
    VideoSink.VideoFrameHandler videoFrameHandler
)

Handles a video input frame.

Must be called after the corresponding stream is signaled.

Parameters
long bufferPresentationTimeUs

The buffer presentation time, in microseconds.
VideoSink.VideoFrameHandler videoFrameHandler

The VideoFrameHandler used to handle the input frame.
Returns
boolean

Whether the frame was handled successfully. If false, the caller can try again later.

initialize

abstract boolean initialize(Format sourceFormat)

Initializes the video sink.

Parameters
Format sourceFormat

The format of the first input video or image.
Returns
boolean

Whether initialization succeeded. If false, the caller should try again later.
Throws
androidx.media3.exoplayer.video.VideoSink.VideoSinkException

If initializing the sink failed.

isEnded

abstract boolean isEnded()

Returns whether all the data has been rendered to the output surface.

This method returns true if the end of the last input stream has been signaled and all the input frames have been rendered. Note that a new input stream can be signaled even when this method returns true (in which case the sink will not be ended anymore).

isInitialized

abstract boolean isInitialized()

Returns whether the video sink is initialized.

isReady

abstract boolean isReady(boolean otherwiseReady)

Returns whether the video sink is able to immediately render media to its output surface from the current position.

The caller should be ready if and only if the video sink is ready.

Parameters
boolean otherwiseReady

Whether the caller is ready except for the video sink.

join

abstract void join(boolean renderNextFrameImmediately)

Joins the video sink to a new stream.

The sink will mask isReady as true for a short time to avoid interrupting an ongoing playback, even if the first frame hasn't yet been rendered to the output surface.

Parameters
boolean renderNextFrameImmediately

Whether the next frame should be rendered as soon as possible or only at its preferred scheduled release time.

onInputStreamChanged

abstract void onInputStreamChanged(
    @VideoSink.InputType int inputType,
    Format format,
    long startPositionUs,
    @VideoSink.FirstFrameReleaseInstruction int firstFrameReleaseInstruction,
    List<Effect> videoEffects
)

Informs the video sink that a new input stream will be queued with the given effects.

Must be called after the sink is initialized.

Parameters
@VideoSink.InputType int inputType

The InputType of the stream.
Format format

The Format of the stream.
long startPositionUs

The start position of the buffer presentation timestamps of the stream, in microseconds.
@VideoSink.FirstFrameReleaseInstruction int firstFrameReleaseInstruction

The FirstFrameReleaseInstruction indicating when to release the stream's first frame.
List<Effect> videoEffects

The < to apply to the new stream.

redraw

abstract void redraw()

Redraws the video effects immediately.

release

abstract void release()

Releases the sink.

render

abstract void render(long positionUs, long elapsedRealtimeUs)

Incrementally renders processed video frames to the output surface.

Parameters
long positionUs

The current playback position, in microseconds.
long elapsedRealtimeUs

elapsedRealtime in microseconds, taken approximately at the time the playback position was positionUs.
Throws
androidx.media3.exoplayer.video.VideoSink.VideoSinkException

If an error occurs during rendering.

setBufferTimestampAdjustmentUs

abstract void setBufferTimestampAdjustmentUs(long bufferTimestampAdjustmentUs)

Sets the buffer timestamp adjustment.

Parameters
long bufferTimestampAdjustmentUs

The timestamp adjustment to add to the buffer presentation timestamps to convert them to frame presentation timestamps, in microseconds.

setChangeFrameRateStrategy

abstract void setChangeFrameRateStrategy(
    @C.VideoChangeFrameRateStrategy int changeFrameRateStrategy
)

Changes the C.VideoChangeFrameRateStrategy used when calling setFrameRate.

The default value is VIDEO_CHANGE_FRAME_RATE_STRATEGY_ONLY_IF_SEAMLESS.

setListener

abstract void setListener(VideoSink.Listener listener, Executor executor)

Sets a Listener on this sink. Callbacks are triggered on the supplied Executor.

Parameters
VideoSink.Listener listener

The Listener.
Executor executor

The Executor to dispatch the callbacks.

setOutputSurfaceInfo

abstract void setOutputSurfaceInfo(Surface outputSurface, Size outputResolution)

Sets the output surface info.

setPlaybackSpeed

abstract void setPlaybackSpeed(@FloatRange(from = 0, fromInclusive = false) float speed)

Sets the playback speed.

setVideoEffects

abstract void setVideoEffects(List<Effect> videoEffects)

Sets video effects to apply immediately.

setVideoFrameMetadataListener

abstract void setVideoFrameMetadataListener(
    VideoFrameMetadataListener videoFrameMetadataListener
)

Sets the VideoFrameMetadataListener.

signalEndOfCurrentInputStream

abstract void signalEndOfCurrentInputStream()

Signals the end of the current input stream.

signalEndOfInput

abstract void signalEndOfInput()

Signals the end of the last input stream.

startRendering

abstract void startRendering()

Starts rendering to the output surface.

stopRendering

abstract void stopRendering()

Stops rendering to the output surface.