Media3 1.5.0 — what’s new?

Posted by Kristina Simakova – Engineering Supervisor

This text is cross-published on Medium

Transformer now helps movement photographs and sooner picture encoding. We’ve additionally simplified the setup for DefaultPreloadManager and ExoPlayer, making it simpler to make use of. However that’s not all! We’ve included a brand new IAMF decoder, a Kotlin listener extension, and simpler Participant optimization by way of delegation.

To be taught extra about all new APIs and bug fixes, try the full launch notes.

Transformer enhancements

Movement picture help

Transformer now helps exporting movement photographs. The movement picture’s picture is exported if the corresponding MediaItem’s picture length is about (see MediaItem.Builder().setImageDurationMs()) In any other case, the movement picture’s video is exported. Observe that the EditedMediaItem’s length shouldn’t be set in both case as it can routinely be set to the corresponding MediaItem’s picture length.

Sooner picture encoding

This launch accelerates image-to-video encoding, due to optimizations in DefaultVideoFrameProcessor.queueInputBitmap(). DefaultVideoFrameProcessor now treats the Bitmap given to queueInputBitmap() as immutable. The GL pipeline will resample and color-convert the enter Bitmap solely as soon as. Because of this, Transformer operations that take giant (e.g. 12 megapixels) photos as enter execute sooner.

AudioEncoderSettings

Much like VideoEncoderSettings, Transformer now helps AudioEncoderSettings which can be utilized to set the specified encoding profile and bitrate.

Edit listing help

Transformer now shifts the primary video body to start out from 0. This fixes A/V sync points in some information the place an edit listing is current.

Unsupported monitor sort logging

This launch contains improved logging for unsupported monitor varieties, offering extra detailed info for troubleshooting and debugging.

Media3 muxer

In one of many earlier releases we added a brand new muxer library which can be utilized to create MP4 container information. The media3 muxer provides help for a variety of audio and video codecs, enabling seamless dealing with of numerous media codecs. This new library additionally brings superior options together with:

The muxer library could be included as a gradle dependency:

implementation ("androidx.media3:media3-muxer:1.5.0")

Media3 muxer with Transformer

To make use of the media3 muxer with Transformer, set an InAppMuxer.Manufacturing facility (which internally wraps media3 muxer) because the muxer manufacturing unit when making a Transformer:

val transformer = Transformer.Builder(context)
    .setMuxerFactory(InAppMuxer.Manufacturing facility.Builder().construct())
    .construct()

Less complicated setup for DefaultPreloadManager and ExoPlayer

With Media3 1.5.0, we added DefaultPreloadManager.Builder, which makes it a lot simpler to construct the preload parts and the participant. Beforehand we requested you to instantiate a number of required parts (RenderersFactory, TrackSelectorFactory, LoadControl, BandwidthMeter and preload / playback Looper) first, and be tremendous cautious on appropriately sharing these parts when injecting them into the DefaultPreloadManager constructor and the ExoPlayer.Builder. With the brand new DefaultPreloadManager.Builder this turns into loads easier:

• Construct a DefaultPreloadManager and ExoPlayer cases with all default parts.

val preloadManagerBuilder = DefaultPreloadManager.Builder()
val preloadManager = preloadManagerBuilder.construct()
val participant = preloadManagerBuilder.buildExoPlayer()

• Construct a DefaultPreloadManager and ExoPlayer cases with customized sharing parts.

val preloadManagerBuilder = DefaultPreloadManager.Builder().setRenderersFactory(customRenderersFactory)
// The ensuing preloadManager makes use of customRenderersFactory
val preloadManager = preloadManagerBuilder.construct()
// The ensuing participant makes use of customRenderersFactory
val participant = preloadManagerBuilder.buildExoPlayer()

• Construct a DefaultPreloadManager and ExoPlayer cases, whereas setting the customized playback-only configurations on the ExoPlayers.

val preloadManagerBuilder = DefaultPreloadManager.Builder()
val preloadManager = preloadManagerBuilder.construct()
// Tune the playback-only configurations
val playerBuilder = ExoPlayer.Builder().setFooEnabled()
// The ensuing participant may have playback function "Foo" enabled
val participant = preloadManagerBuilder.buildExoPlayer(playerBuilder)

Preloading the subsequent playlist merchandise

We’ve added the power to preload the subsequent merchandise within the playlist of ExoPlayer. By default, playlist preloading is disabled however could be enabled by setting the length which needs to be preloaded to reminiscence:

participant.preloadConfiguration =
    PreloadConfiguration(/* targetPreloadDurationUs= */ 5_000_000L)

With the PreloadConfiguration above, the participant tries to preload 5 seconds of media for the subsequent merchandise within the playlist. Preloading is simply began when no media is being loaded that’s required for the continued playback. This manner preloading doesn’t compete for bandwidth with the first playback.

When enabled, preloading may also help decrease be part of latency when a person skips to the subsequent merchandise earlier than the playback buffer reaches the subsequent merchandise. The primary interval of the subsequent window is ready and video, audio and textual content samples are preloaded into its pattern queues. The preloaded interval is later queued into the participant with preloaded samples instantly out there and able to be fed to the codec for rendering.

As soon as opted-in, playlist preloading could be turned off once more by utilizing PreloadConfiguration.DEFAULT to disable playlist preloading:

participant.preloadConfiguration = PreloadConfiguration.DEFAULT

New IAMF decoder and Kotlin listener extension

The 1.5.0 launch features a new media3-decoder-iamf module, which permits playback of IAMF immersive audio tracks in MP4 information. Apps wanting to do this out might want to construct the libiamf decoder domestically. See the media3 README for full directions.

implementation ("androidx.media3:media3-decoder-iamf:1.5.0")

This launch additionally features a new media3-common-ktx module, a house for Kotlin-specific performance. The primary model of this module accommodates a droop operate that lets the caller take heed to Participant.Listener.onEvents. This can be a constructing block that’s utilized by the upcoming media3-ui-compose module (launching with media3 1.6.0) to energy a Jetpack Compose playback UI.

implementation ("androidx.media3:media3-common-ktx:1.5.0")

Simpler Participant customization by way of delegation

Media3 has supplied a ForwardingPlayer implementation since model 1.0.0, and now we have beforehand prompt that apps ought to use it after they wish to customise the best way sure Participant operations work, by utilizing the decorator sample. One quite common use-case is to permit or disallow sure participant instructions (with the intention to present/disguise sure buttons in a UI). Sadly, doing this appropriately with ForwardingPlayer is surprisingly exhausting and error-prone, as a result of it’s a must to persistently override a number of strategies, and deal with the listener as properly. The instance code to exhibit how fiddly that is too lengthy for this weblog, so we’ve put it in a gist as an alternative.

To be able to make these types of customizations simpler, 1.5.0 features a new ForwardingSimpleBasePlayer, which builds on the consistency ensures supplied by SimpleBasePlayer to make it simpler to create constant Participant implementations following the decorator sample. The identical command-modifying Participant is now a lot easier to implement:

class PlayerWithoutSeekToNext(participant: Participant) : ForwardingSimpleBasePlayer(participant) {
  override enjoyable getState(): State {
    val state = tremendous.getState()
    return state
      .buildUpon()
      .setAvailableCommands(
        state.availableCommands.buildUpon().take away(COMMAND_SEEK_TO_NEXT).construct()
      )
      .construct()
  }

  // We need not override handleSeek, as a result of it's assured to not be known as for
  // COMMAND_SEEK_TO_NEXT since we have marked that command unavailable.
}

MediaSession: Command button for media gadgets

Command buttons for media gadgets permit a session app to declare instructions supported by sure media gadgets that then could be conveniently displayed and executed by a MediaController or MediaBrowser:

You will discover the detailed documentation on android.developer.com.

That is the Media3 equal of the legacy “customized browse actions” API, with which Media3 is absolutely interoperable. In contrast to the legacy API, command buttons for media gadgets don’t require a MediaLibraryService however are a function of the Media3 MediaSession as an alternative. Therefore they’re out there for MediaController and MediaBrowser in the identical method.

In case you encounter any points, have function requests, or wish to share suggestions, please tell us utilizing the Media3 challenge tracker on GitHub. We stay up for listening to from you!