MediaMuxer
class MediaMuxer
kotlin.Any | |
↳ | android.media.MediaMuxer |
MediaMuxer facilitates muxing elementary streams. Currently MediaMuxer supports MP4, Webm and 3GP file as the output. It also supports muxing B-frames in MP4 since Android Nougat.
It is generally used like this:
MediaMuxer muxer = new MediaMuxer("temp.mp4", OutputFormat.MUXER_OUTPUT_MPEG_4); // More often, the MediaFormat will be retrieved from MediaCodec.getOutputFormat() // or MediaExtractor.getTrackFormat(). MediaFormat audioFormat = new MediaFormat(...); MediaFormat videoFormat = new MediaFormat(...); int audioTrackIndex = muxer.addTrack(audioFormat); int videoTrackIndex = muxer.addTrack(videoFormat); ByteBuffer inputBuffer = ByteBuffer.allocate(bufferSize); boolean finished = false; BufferInfo bufferInfo = new BufferInfo(); muxer.start(); while(!finished) { // getInputBuffer() will fill the inputBuffer with one frame of encoded // sample from either MediaCodec or MediaExtractor, set isAudioSample to // true when the sample is audio data, set up all the fields of bufferInfo, // and return true if there are no more samples. finished = getInputBuffer(inputBuffer, isAudioSample, bufferInfo); if (!finished) { int currentTrackIndex = isAudioSample ? audioTrackIndex : videoTrackIndex; muxer.writeSampleData(currentTrackIndex, inputBuffer, bufferInfo); } }; muxer.stop(); muxer.release();
Metadata Track
Per-frame metadata carries information that correlates with video or audio to facilitate offline processing. For example, gyro signals from the sensor can help video stabilization when doing offline processing. Metadata tracks are only supported when multiplexing to the MP4 container format. When adding a new metadata track, the MIME type format must start with prefix "application/" (for example, "application/gyro"). The format of the metadata is application-defined. Metadata timestamps must be in the same time base as video and audio timestamps. The generated MP4 file uses TextMetaDataSampleEntry (defined in section 12.3.3.2 of the ISOBMFF specification) to signal the metadata's MIME type.
MediaMuxer muxer = new MediaMuxer("temp.mp4", OutputFormat.MUXER_OUTPUT_MPEG_4); // SetUp Video/Audio Tracks. MediaFormat audioFormat = new MediaFormat(...); MediaFormat videoFormat = new MediaFormat(...); int audioTrackIndex = muxer.addTrack(audioFormat); int videoTrackIndex = muxer.addTrack(videoFormat); // Setup Metadata Track MediaFormat metadataFormat = new MediaFormat(...); metadataFormat.setString(KEY_MIME, "application/gyro"); int metadataTrackIndex = muxer.addTrack(metadataFormat); muxer.start(); while(..) { // Allocate bytebuffer and write gyro data(x,y,z) into it. ByteBuffer metaData = ByteBuffer.allocate(bufferSize); metaData.putFloat(x); metaData.putFloat(y); metaData.putFloat(z); BufferInfo metaInfo = new BufferInfo(); // Associate this metadata with the video frame by setting // the same timestamp as the video frame. metaInfo.presentationTimeUs = currentVideoTrackTimeUs; metaInfo.offset = 0; metaInfo.flags = 0; metaInfo.size = bufferSize; muxer.writeSampleData(metadataTrackIndex, metaData, metaInfo); }; muxer.stop(); muxer.release(); }
Features and API History
The following table summarizes the feature support in different API version and containers. For API version numbers, see android.os.Build.VERSION_CODES
.
Symbol | Meaning |
---|---|
● | Supported |
○ | Not supported |
▧ | Supported in MP4/WebM/3GP |
⁕ | Only Supported in MP4 |
Feature | SDK Version | |||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26+ | ||||||||||||||||
MP4 container | ● | ● | ● | ● | ● | ● | ● | ● | ● | |||||||||||||||
WebM container | ○ | ○ | ○ | ● | ● | ● | ● | ● | ● | |||||||||||||||
3GP container | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ● | |||||||||||||||
Muxing B-Frames(bi-directional predicted frames) | ○ | ○ | ○ | ○ | ○ | ○ | ⁕ | ⁕ | ⁕ | |||||||||||||||
Muxing Single Video/Audio Track | ▧ | ▧ | ▧ | ▧ | ▧ | ▧ | ▧ | ▧ | ▧ | |||||||||||||||
Muxing Multiple Video/Audio Tracks | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ⁕ | |||||||||||||||
Muxing Metadata Tracks | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ⁕ |
Summary
Nested classes | |
---|---|
Defines the output format. |
Public constructors | |
---|---|
MediaMuxer(path: String, format: Int) Creates a media muxer that writes to the specified path. |
|
MediaMuxer(fd: FileDescriptor, format: Int) Creates a media muxer that writes to the specified FileDescriptor. |
Public methods | |
---|---|
Int |
addTrack(format: MediaFormat) Adds a track with the specified format. |
Unit |
release() Make sure you call this when you're done to free up any resources instead of relying on the garbage collector to do this for you at some point in the future. |
Unit |
setLocation(latitude: Float, longitude: Float) Set and store the geodata (latitude and longitude) in the output file. |
Unit |
setOrientationHint(degrees: Int) Sets the orientation hint for output video playback. |
Unit |
start() Starts the muxer. |
Unit |
stop() Stops the muxer. |
Unit |
writeSampleData(trackIndex: Int, byteBuf: ByteBuffer, bufferInfo: MediaCodec.BufferInfo) Writes an encoded sample into the muxer. |
Protected methods | |
---|---|
Unit |
finalize() |
Public constructors
MediaMuxer
MediaMuxer(
path: String,
format: Int)
Creates a media muxer that writes to the specified path.
The caller must not use the file path
before calling stop
.
Parameters | |
---|---|
path |
String: The path of the output media file. This value cannot be null . |
format |
Int: The format of the output media file. Value is android.media.MediaMuxer.OutputFormat#MUXER_OUTPUT_MPEG_4 , android.media.MediaMuxer.OutputFormat#MUXER_OUTPUT_WEBM , android.media.MediaMuxer.OutputFormat#MUXER_OUTPUT_3GPP , android.media.MediaMuxer.OutputFormat#MUXER_OUTPUT_HEIF , or android.media.MediaMuxer.OutputFormat#MUXER_OUTPUT_OGG |
Exceptions | |
---|---|
java.lang.IllegalArgumentException |
if path is invalid or format is not supported. |
java.io.IOException |
if an error occurs while opening or creating the output file. |
MediaMuxer
MediaMuxer(
fd: FileDescriptor,
format: Int)
Creates a media muxer that writes to the specified FileDescriptor.
The caller must not use the file referenced by the specified fd
before calling stop
.
It is the caller's responsibility to close the file descriptor, which is safe to do so as soon as this call returns.
Parameters | |
---|---|
fd |
FileDescriptor: The FileDescriptor of the output media file. If format is OutputFormat#MUXER_OUTPUT_WEBM , fd must be open in read-write mode. Otherwise, write mode is sufficient, but read-write is also accepted. This value cannot be null . |
format |
Int: The format of the output media file. Value is android.media.MediaMuxer.OutputFormat#MUXER_OUTPUT_MPEG_4 , android.media.MediaMuxer.OutputFormat#MUXER_OUTPUT_WEBM , android.media.MediaMuxer.OutputFormat#MUXER_OUTPUT_3GPP , android.media.MediaMuxer.OutputFormat#MUXER_OUTPUT_HEIF , or android.media.MediaMuxer.OutputFormat#MUXER_OUTPUT_OGG |
Exceptions | |
---|---|
java.lang.IllegalArgumentException |
if format is not supported, or if fd is not open in the expected mode. |
java.io.IOException |
if an error occurs while performing an IO operation. |
Public methods
addTrack
fun addTrack(format: MediaFormat): Int
Adds a track with the specified format.
The following table summarizes support for specific format keys across android releases. Keys marked with '+:' are required.
OS Version(s) | MediaFormat keys used for |
||
---|---|---|---|
All Tracks | Audio Tracks | Video Tracks | |
android.os.Build.VERSION_CODES#JELLY_BEAN_MR2 |
+: MediaFormat#KEY_MIME |
+: MediaFormat#KEY_SAMPLE_RATE ,+: MediaFormat#KEY_CHANNEL_COUNT ,+: codec-specific dataAAC |
+: MediaFormat#KEY_WIDTH ,+: MediaFormat#KEY_HEIGHT ,no KEY_ROTATION , use setOrientationHint() .mp4,+: codec-specific dataAVC, MPEG4 |
android.os.Build.VERSION_CODES#KITKAT |
|||
android.os.Build.VERSION_CODES#KITKAT_WATCH |
|||
android.os.Build.VERSION_CODES#LOLLIPOP |
as above, plus +: codec-specific dataVorbis & .webm |
||
android.os.Build.VERSION_CODES#LOLLIPOP_MR1 |
|||
android.os.Build.VERSION_CODES#M |
as above, plusMediaFormat#KEY_BIT_RATE AAC |
||
android.os.Build.VERSION_CODES#N |
as above, plusMediaFormat#KEY_BIT_RATE MPEG4,MediaFormat#KEY_HDR_STATIC_INFO , .webm,MediaFormat#KEY_COLOR_STANDARD ,MediaFormat#KEY_COLOR_TRANSFER ,MediaFormat#KEY_COLOR_RANGE ,+: codec-specific dataHEVC, codec-specific dataVP9 |
||
Notes: |
|||
Note that the codec-specific data for the track must be specified using this method. Furthermore, codec-specific data must not be passed/specified via the |
The following table summarizes codec support for containers across android releases:
OS Version(s) | Codec support | ||
---|---|---|---|
MP4 | WEBM | ||
android.os.Build.VERSION_CODES#JELLY_BEAN_MR2 |
AAC ,NB-AMR ,WB-AMR ,H.263 ,MPEG-4 ,AVC (H.264) |
Not supported | |
android.os.Build.VERSION_CODES#KITKAT |
|||
android.os.Build.VERSION_CODES#KITKAT_WATCH |
|||
android.os.Build.VERSION_CODES#LOLLIPOP |
Vorbis ,VP8 |
||
android.os.Build.VERSION_CODES#LOLLIPOP_MR1 |
|||
android.os.Build.VERSION_CODES#M |
|||
android.os.Build.VERSION_CODES#N |
as above, plusHEVC (H.265) |
as above, plusVP9 |
Parameters | |
---|---|
format |
MediaFormat: The media format for the track. This must not be an empty MediaFormat. This value cannot be null . |
Return | |
---|---|
Int |
The track index for this newly added track, and it should be used in the writeSampleData . |
Exceptions | |
---|---|
java.lang.IllegalArgumentException |
if format is invalid. |
java.lang.IllegalStateException |
if muxer is in the wrong state. |
release
fun release(): Unit
Make sure you call this when you're done to free up any resources instead of relying on the garbage collector to do this for you at some point in the future.
setLocation
fun setLocation(
latitude: Float,
longitude: Float
): Unit
Set and store the geodata (latitude and longitude) in the output file. This method should be called before start
. The geodata is stored in udta box if the output format is OutputFormat#MUXER_OUTPUT_MPEG_4
, and is ignored for other output formats. The geodata is stored according to ISO-6709 standard.
Parameters | |
---|---|
latitude |
Float: Latitude in degrees. Its value must be in the range [-90, 90]. |
longitude |
Float: Longitude in degrees. Its value must be in the range [-180, 180]. |
Exceptions | |
---|---|
java.lang.IllegalArgumentException |
If the given latitude or longitude is out of range. |
java.lang.IllegalStateException |
If this method is called after start . |
setOrientationHint
fun setOrientationHint(degrees: Int): Unit
Sets the orientation hint for output video playback.
This method should be called before start
. Calling this method will not rotate the video frame when muxer is generating the file, but add a composition matrix containing the rotation angle in the output video if the output format is OutputFormat#MUXER_OUTPUT_MPEG_4
so that a video player can choose the proper orientation for playback. Note that some video players may choose to ignore the composition matrix in a video during playback. By default, the rotation degree is 0.
Parameters | |
---|---|
degrees |
Int: the angle to be rotated clockwise in degrees. The supported angles are 0, 90, 180, and 270 degrees. |
Exceptions | |
---|---|
java.lang.IllegalArgumentException |
if degree is not supported. |
java.lang.IllegalStateException |
If this method is called after start . |
start
fun start(): Unit
Starts the muxer.
Make sure this is called after addTrack
and before writeSampleData
.
Exceptions | |
---|---|
java.lang.IllegalStateException |
If this method is called after start or Muxer is released |
stop
fun stop(): Unit
Stops the muxer.
Once the muxer stops, it can not be restarted.
Exceptions | |
---|---|
java.lang.IllegalStateException |
if muxer is in the wrong state. |
writeSampleData
fun writeSampleData(
trackIndex: Int,
byteBuf: ByteBuffer,
bufferInfo: MediaCodec.BufferInfo
): Unit
Writes an encoded sample into the muxer.
The application needs to make sure that the samples are written into the right tracks. Also, it needs to make sure the samples for each track are written in chronological order (e.g. in the order they are provided by the encoder.)
For MPEG4 media format, the duration of the last sample in a track can be set by passing an additional empty buffer(bufferInfo.size = 0) with MediaCodec.BUFFER_FLAG_END_OF_STREAM flag and a suitable presentation timestamp set in bufferInfo parameter as the last sample of that track. This last sample's presentation timestamp shall be a sum of the presentation timestamp and the duration preferred for the original last sample. If no explicit END_OF_STREAM sample was passed, then the duration of the last sample would be the same as that of the sample before that.
Parameters | |
---|---|
byteBuf |
ByteBuffer: The encoded sample. This value cannot be null . |
trackIndex |
Int: The track index for this sample. |
bufferInfo |
MediaCodec.BufferInfo: The buffer information related to this sample. This value cannot be null . |
Exceptions | |
---|---|
java.lang.IllegalArgumentException |
if trackIndex, byteBuf or bufferInfo is invalid. |
java.lang.IllegalStateException |
if muxer is in wrong state. MediaMuxer uses the flags provided in MediaCodec.BufferInfo , to signal sync frames. |
Protected methods
finalize
protected fun finalize(): Unit
Exceptions | |
---|---|
java.lang.Throwable |
the Exception raised by this method |