@UnstableApi
interface VideoSink

A sink that consumes decoded video frames and images.

Summary

Nested types
@Documented
@Retention(value = RetentionPolicy.SOURCE)
@Target(value = TYPE_USE)
@UnstableApi
@IntDef(value = )
annotation VideoSink.FirstFrameReleaseInstruction

The instruction provided when the stream changes for releasing the first frame.
@Documented
@Retention(value = RetentionPolicy.SOURCE)
@Target(value = TYPE_USE)
@IntDef(value = )
annotation VideoSink.InputType

Specifies how the input frames are made available to the video sink.

Listener for VideoSink events.

Handler for a video frame.

Thrown by VideoSink implementations.

Constants
const Int

Input frames come from a Bitmap.
const Int

Input frames come from a surface.
const Int

Instructs to release the first frame as soon as possible.
const Int

Instructs to release the first frame when the playback position reaches the stream start position.
const Int

Instructs to release the first frame when rendering starts.

Public functions
Unit

Allows the sink to release the first frame even if rendering is not started.
Unit

Clears the set output surface info.
Unit
flush(resetPosition: Boolean)

Flushes the video sink.
Surface!

Returns the input Surface where the video sink consumes input frames from.
Boolean
handleInputBitmap(
    inputBitmap: Bitmap!,
    bufferTimestampIterator: TimestampIterator!
)

Handles an input Bitmap.
Boolean
handleInputFrame(
    bufferPresentationTimeUs: Long,
    videoFrameHandler: VideoSink.VideoFrameHandler!
)

Handles a video input frame.
Boolean
initialize(sourceFormat: Format!)

Initializes the video sink.
Boolean

Returns whether all the data has been rendered to the output surface.
Boolean

Returns whether the video sink is initialized.
Boolean
isReady(otherwiseReady: Boolean)

Returns whether the video sink is able to immediately render media to its output surface from the current position.
Unit
join(renderNextFrameImmediately: Boolean)

Joins the video sink to a new stream.
Unit
onInputStreamChanged(
    @VideoSink.InputType inputType: Int,
    format: Format!,
    startPositionUs: Long,
    @VideoSink.FirstFrameReleaseInstruction firstFrameReleaseInstruction: Int,
    videoEffects: (Mutable)List<Effect!>!
)

Informs the video sink that a new input stream will be queued with the given effects.
Unit

Redraws the video effects immediately.
Unit

Releases the sink.
Unit
render(positionUs: Long, elapsedRealtimeUs: Long)

Incrementally renders processed video frames to the output surface.
Unit
setBufferTimestampAdjustmentUs(bufferTimestampAdjustmentUs: Long)

Sets the buffer timestamp adjustment.
Unit

Changes the C.VideoChangeFrameRateStrategy used when calling setFrameRate.
Unit
setListener(listener: VideoSink.Listener!, executor: Executor!)

Sets a Listener on this sink.
Unit
setOutputSurfaceInfo(outputSurface: Surface!, outputResolution: Size!)

Sets the output surface info.
Unit
setPlaybackSpeed(speed: @FloatRange(from = 0, fromInclusive = false) Float)

Sets the playback speed.
Unit
setVideoEffects(videoEffects: (Mutable)List<Effect!>!)

Sets video effects to apply immediately.
Unit
setVideoFrameMetadataListener(
    videoFrameMetadataListener: VideoFrameMetadataListener!
)

Sets the VideoFrameMetadataListener.
Unit

Signals the end of the current input stream.
Unit

Signals the end of the last input stream.
Unit

Starts rendering to the output surface.
Unit

Stops rendering to the output surface.

Constants

INPUT_TYPE_BITMAP

const val INPUT_TYPE_BITMAP = 2: Int

Input frames come from a Bitmap.

INPUT_TYPE_SURFACE

const val INPUT_TYPE_SURFACE = 1: Int

Input frames come from a surface.

RELEASE_FIRST_FRAME_IMMEDIATELY

const val RELEASE_FIRST_FRAME_IMMEDIATELY = 0: Int

Instructs to release the first frame as soon as possible.

RELEASE_FIRST_FRAME_WHEN_PREVIOUS_STREAM_PROCESSED

const val RELEASE_FIRST_FRAME_WHEN_PREVIOUS_STREAM_PROCESSED = 2: Int

Instructs to release the first frame when the playback position reaches the stream start position.

RELEASE_FIRST_FRAME_WHEN_STARTED

const val RELEASE_FIRST_FRAME_WHEN_STARTED = 1: Int

Instructs to release the first frame when rendering starts.

Public functions

allowReleaseFirstFrameBeforeStarted

fun allowReleaseFirstFrameBeforeStarted(): Unit

Allows the sink to release the first frame even if rendering is not started.

This is used to update the FirstFrameReleaseInstruction of the stream that is currently being processed.

clearOutputSurfaceInfo

fun clearOutputSurfaceInfo(): Unit

Clears the set output surface info.

flush

fun flush(resetPosition: Boolean): Unit

Flushes the video sink.

After calling this method, any frames stored inside the video sink are discarded.

Parameters
resetPosition: Boolean

Whether to reset the current position.

getInputSurface

fun getInputSurface(): Surface!

Returns the input Surface where the video sink consumes input frames from.

Must be called after the sink is initialized.

handleInputBitmap

fun handleInputBitmap(
    inputBitmap: Bitmap!,
    bufferTimestampIterator: TimestampIterator!
): Boolean

Handles an input Bitmap.

Must be called after the corresponding stream is signaled.

Parameters
inputBitmap: Bitmap!

The Bitmap to queue to the video sink.
bufferTimestampIterator: TimestampIterator!

The buffer presentation times within the current stream that the bitmap should be shown at. The timestamps should be monotonically increasing.
Returns
Boolean

Whether the bitmap was queued successfully. If false, the caller can try again later.

handleInputFrame

fun handleInputFrame(
    bufferPresentationTimeUs: Long,
    videoFrameHandler: VideoSink.VideoFrameHandler!
): Boolean

Handles a video input frame.

Must be called after the corresponding stream is signaled.

Parameters
bufferPresentationTimeUs: Long

The buffer presentation time, in microseconds.
videoFrameHandler: VideoSink.VideoFrameHandler!

The VideoFrameHandler used to handle the input frame.
Returns
Boolean

Whether the frame was handled successfully. If false, the caller can try again later.

initialize

fun initialize(sourceFormat: Format!): Boolean

Initializes the video sink.

Parameters
sourceFormat: Format!

The format of the first input video or image.
Returns
Boolean

Whether initialization succeeded. If false, the caller should try again later.
Throws
androidx.media3.exoplayer.video.VideoSink.VideoSinkException

If initializing the sink failed.

isEnded

fun isEnded(): Boolean

Returns whether all the data has been rendered to the output surface.

This method returns true if the end of the last input stream has been signaled and all the input frames have been rendered. Note that a new input stream can be signaled even when this method returns true (in which case the sink will not be ended anymore).

isInitialized

fun isInitialized(): Boolean

Returns whether the video sink is initialized.

isReady

fun isReady(otherwiseReady: Boolean): Boolean

Returns whether the video sink is able to immediately render media to its output surface from the current position.

The caller should be ready if and only if the video sink is ready.

Parameters
otherwiseReady: Boolean

Whether the caller is ready except for the video sink.

join

fun join(renderNextFrameImmediately: Boolean): Unit

Joins the video sink to a new stream.

The sink will mask isReady as true for a short time to avoid interrupting an ongoing playback, even if the first frame hasn't yet been rendered to the output surface.

Parameters
renderNextFrameImmediately: Boolean

Whether the next frame should be rendered as soon as possible or only at its preferred scheduled release time.

onInputStreamChanged

fun onInputStreamChanged(
    @VideoSink.InputType inputType: Int,
    format: Format!,
    startPositionUs: Long,
    @VideoSink.FirstFrameReleaseInstruction firstFrameReleaseInstruction: Int,
    videoEffects: (Mutable)List<Effect!>!
): Unit

Informs the video sink that a new input stream will be queued with the given effects.

Must be called after the sink is initialized.

Parameters
@VideoSink.InputType inputType: Int

The InputType of the stream.
format: Format!

The Format of the stream.
startPositionUs: Long

The start position of the buffer presentation timestamps of the stream, in microseconds.
@VideoSink.FirstFrameReleaseInstruction firstFrameReleaseInstruction: Int

The FirstFrameReleaseInstruction indicating when to release the stream's first frame.
videoEffects: (Mutable)List<Effect!>!

The < to apply to the new stream.

redraw

fun redraw(): Unit

Redraws the video effects immediately.

release

fun release(): Unit

Releases the sink.

render

fun render(positionUs: Long, elapsedRealtimeUs: Long): Unit

Incrementally renders processed video frames to the output surface.

Parameters
positionUs: Long

The current playback position, in microseconds.
elapsedRealtimeUs: Long

elapsedRealtime in microseconds, taken approximately at the time the playback position was positionUs.
Throws
androidx.media3.exoplayer.video.VideoSink.VideoSinkException

If an error occurs during rendering.

setBufferTimestampAdjustmentUs

fun setBufferTimestampAdjustmentUs(bufferTimestampAdjustmentUs: Long): Unit

Sets the buffer timestamp adjustment.

Parameters
bufferTimestampAdjustmentUs: Long

The timestamp adjustment to add to the buffer presentation timestamps to convert them to frame presentation timestamps, in microseconds.

setChangeFrameRateStrategy

fun setChangeFrameRateStrategy(
    @C.VideoChangeFrameRateStrategy changeFrameRateStrategy: Int
): Unit

Changes the C.VideoChangeFrameRateStrategy used when calling setFrameRate.

The default value is VIDEO_CHANGE_FRAME_RATE_STRATEGY_ONLY_IF_SEAMLESS.

setListener

fun setListener(listener: VideoSink.Listener!, executor: Executor!): Unit

Sets a Listener on this sink. Callbacks are triggered on the supplied Executor.

Parameters
listener: VideoSink.Listener!

The Listener.
executor: Executor!

The Executor to dispatch the callbacks.

setOutputSurfaceInfo

fun setOutputSurfaceInfo(outputSurface: Surface!, outputResolution: Size!): Unit

Sets the output surface info.

setPlaybackSpeed

fun setPlaybackSpeed(speed: @FloatRange(from = 0, fromInclusive = false) Float): Unit

Sets the playback speed.

setVideoEffects

fun setVideoEffects(videoEffects: (Mutable)List<Effect!>!): Unit

Sets video effects to apply immediately.

setVideoFrameMetadataListener

fun setVideoFrameMetadataListener(
    videoFrameMetadataListener: VideoFrameMetadataListener!
): Unit

Sets the VideoFrameMetadataListener.

signalEndOfCurrentInputStream

fun signalEndOfCurrentInputStream(): Unit

Signals the end of the current input stream.

signalEndOfInput

fun signalEndOfInput(): Unit

Signals the end of the last input stream.

startRendering

fun startRendering(): Unit

Starts rendering to the output surface.

stopRendering

fun stopRendering(): Unit

Stops rendering to the output surface.