Skip to content

[video_player] Adds audio track metadata fetching and audio track selection feature #9925

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
nateshmbhat wants to merge 75 commits into
base: main
Choose a base branch
from
Open

[video_player] Adds audio track metadata fetching and audio track selection feature #9925

nateshmbhat wants to merge 75 commits into from

Conversation

@nateshmbhat
Copy link
Contributor

@nateshmbhat nateshmbhat commented Aug 29, 2025 •
edited
Loading

Description

This PR adds comprehensive audio track retrieval and selection support to the video_player package, enabling developers to access detailed information about available audio tracks and switch between them during playback.

Breakout PRs:

  1. Platform interface : [video_player] Add audio track management support to platform interface #10171
  2. Andrioid : feat(android): add audio track selection for ExoPlayer #10312
  3. ios : feat(ios): add audio track selection support for AVFoundation #10313

Changes Made

Core Features

  • Added VideoAudioTrack model with comprehensive metadata fields: id, label, language, isSelected, bitrate, sampleRate, channelCount, codec
  • Added getAudioTracks() method to retrieve all available audio tracks with real metadata
  • Added selectAudioTrack() method to switch between audio tracks during playback
  • Updated VideoPlayerController to expose audio track functionality

Platform Implementations

  • Android:
    • Real metadata extraction using ExoPlayer's getCurrentTracks() API
    • Robust track selection using TrackSelectionOverride with proper error handling
    • Support for multiple audio formats (AAC, AC3, EAC3, MP3, etc.)
  • iOS:
    • Metadata extraction from AVFoundation using AVAssetTrack for regular videos
    • HLS stream support using AVMediaSelectionGroup for adaptive streams
    • Proper track selection for both asset tracks and media selection options

Technical Infrastructure

  • Updated Pigeon interfaces for both Android and iOS with new data structures:
    • AudioTrackMessage, ExoPlayerAudioTrackData, AssetAudioTrackData, MediaSelectionAudioTrackData, NativeAudioTrackData
  • Enhanced platform interface with new methods and data classes
  • Has native unit tests for both Android and iOS platforms
  • Created demo screen showcasing audio track functionality with interactive UI

Demo Features

  • Interactive video player with audio track selection
  • Real-time metadata display (bitrate, sample rate, channels, codec)
  • Support for multiple video sources including HLS streams
  • Visual indicators for currently selected tracks

Related Issues

#59437 - Add audio track metadata support to video_player

Testing

  • Added native unit tests for both Android and iOS
  • Tested with various video formats and HLS streams
  • Verified backward compatibility with existing functionality
  • Demo screen widget for manual testing and validation

Breaking Changes

None - all changes are additive and backward compatible.

Pre-Review Checklist

  • I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
  • I read the [Tree Hygiene] page, which explains my responsibilities.
  • I read and followed the [relevant style guides] and ran [the auto-formatter].
  • I signed the [CLA].
  • The title of the PR starts with the name of the package surrounded by square brackets, e.g. [video_player]
  • I [linked to at least one issue that this PR fixes] in the description above.
  • I updated pubspec.yaml with an appropriate new version according to the [pub versioning philosophy], or I have commented below to indicate which [version change exemption] this PR falls under[^1].
  • I updated CHANGELOG.md to add a description of the change, [following repository CHANGELOG style], or I have commented below to indicate which [CHANGELOG exemption] this PR falls under[^1].
  • I updated/added any relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or I have commented below to indicate which [test exemption] this PR falls under[^1].
  • All existing and new tests are passing.

@nateshmbhat nateshmbhat mentioned this pull request Aug 29, 2025
Closed
11 tasks
gemini-code-assist[bot]
gemini-code-assist bot reviewed Aug 29, 2025
View reviewed changes
Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a valuable feature for fetching and selecting audio tracks in the video_player package. The changes are comprehensive, touching the platform interface, Android (ExoPlayer) and iOS (AVFoundation) implementations, and adding a new demo screen.

My review has identified several critical and high-severity issues that should be addressed:

  • The new native unit tests for both Android and iOS appear to be broken due to type mismatches and incorrect method calls, and will not compile in their current state.
  • The Dart implementation code for both Android and iOS contains unsafe null assertions on data coming from the native side, which could lead to runtime crashes.
  • The Android implementation has been updated to use a milestone version of Gradle, which is not recommended for production packages.
  • The new audio track selection feature has not been implemented for the web platform, which will lead to UnimplementedError on web.
  • There is a minor issue in the new example app where context is used after an await without checking if the widget is still mounted.

Addressing these points will significantly improve the robustness and completeness of this new feature.

@nateshmbhat
Copy link
Contributor Author

nateshmbhat commented Aug 30, 2025

I updated native andorid tests and ran the tests and verified that all native android tests are passing.

@nateshmbhat
Copy link
Contributor Author

nateshmbhat commented Sep 1, 2025

@stuartmorgan-g @ash2moon can you please comment on this PR and lemme know if there are any concerns ?

@stuartmorgan-g stuartmorgan-g added triage-ios Should be looked at in iOS triage triage-android Should be looked at in Android triage labels Sep 2, 2025
@nateshmbhat
fix(video_player_avfoundation): update audio track selection for macO…
08e20d3 
…S compatibility

- Added macOS 10.13 availability check alongside iOS 11.0
- Updated to use currentMediaSelection property instead of deprecated selectedMediaOptionInMediaSelectionGroup method
@nateshmbhat
Merge branch 'main' into develop
c63b050
This was referenced Nov 7, 2025
Closed
Closed
@nateshmbhat
refactor(video_player): change audio track selection from string ID t…
e3df547 
…o separate indices

- Replaced concatenated string trackId with separate groupIndex and trackIndex parameters in Android implementation
- Updated iOS/macOS to use trackType and numeric trackId instead of prefixed string format
- Modified Pigeon message definitions and generated code to support new parameter structure
@nateshmbhat
Copy link
Contributor Author

nateshmbhat commented Nov 7, 2025 •
edited
Loading

Have addressed all the major concerns in this PR , can we please have the platform PRs merged ? If any furthur changes are needed, pls let me know at the earliest so that i can quickly push the changes and we can take this forward sooner.

cc: @stuartmorgan-g @ash2moon @LongCatIsLooong

@stuartmorgan-g
Copy link
Collaborator

stuartmorgan-g commented Nov 7, 2025

Have addressed all the major concerns in this PR , can we please have the platform PRs merged ? If any furthur changes are needed, pls let me know at the earliest so that i can quickly push the changes and we can take this forward sooner.

I'm not sure why you are pinging people here. This PR can't land until the sub-PRs have landed, and the sub PRs are still under review. Those PRs will be landed when they have been reviewed and approved.

@reidbaker reidbaker requested review from camsim99 and removed request for ash2moon and ezamagni-tg November 11, 2025 18:08
nateshmbhat
nateshmbhat commented Nov 12, 2025
View reviewed changes
packages/video_player/video_player/lib/video_player.dart
/// Throws an exception if the video player is disposed.
Future<List<VideoAudioTrack>> getAudioTracks() async {
if (_isDisposed) {
throw Exception('VideoPlayerController is disposed');
Copy link
Contributor Author

@nateshmbhat nateshmbhat Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be throwing an exception if the video is already disposed ? or should be just return like how its just returned in many of the other methods ?

@nateshmbhat
refactor(video_player_android): improve audio track selection error h…
a7118ee 
…andling and remove redundant tests

- Replace try-catch with explicit bounds checking in selectAudioTrack method
- Add separate validation for groupIndex and trackIndex bounds
- Improve error messages to distinguish between different validation failures
- Remove AudioTracksTest.java as tests are now covered in VideoPlayerTest.java
- Add comprehensive test coverage for audio track selection edge cases
@okorohelijah okorohelijah removed the request for review from LongCatIsLooong November 24, 2025 19:42
@hellohuanlin hellohuanlin requested review from LongCatIsLooong and removed request for hellohuanlin December 1, 2025 22:53
philipgiuliani added a commit to kim-company/flutter-player that referenced this pull request Dec 5, 2025
@philipgiuliani
feat(video_player): Add controller-level audio track API
4e4ee72 
Cherry-picked from upstream PR flutter#9925:
- Add getAudioTracks() to VideoPlayerController
- Add selectAudioTrack() to VideoPlayerController
- Add isAudioTrackSupportAvailable() for feature detection
- Export VideoAudioTrack from platform interface
- Bump version to 2.10.0

This provides the public API that users call to:
1. Check if audio track selection is supported
2. Get list of available audio tracks with metadata
3. Select a specific audio track by ID

Source: flutter#9925
@ezamagni ezamagni mentioned this pull request Dec 6, 2025
Open
13 tasks
auto-submit bot pushed a commit that referenced this pull request Dec 16, 2025
@nateshmbhat
feat(android): add audio track selection for ExoPlayer (#10312)
57725eb 
- Implemented getAudioTracks() and selectAudioTrack() methods for Android video player

Android PR for : #9925

## Pre-Review Checklist
@camsim99
Copy link
Contributor

camsim99 commented Dec 22, 2025

@nateshmbhat I'll take a look at the Android changes again once you port them over from the split out PR. Thanks!

@nateshmbhat
Copy link
Contributor Author

nateshmbhat commented Dec 25, 2025

@camsim99 i have ported the android changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mdebbar mdebbar mdebbar approved these changes

@tarrinneal tarrinneal Awaiting requested review from tarrinneal tarrinneal is a code owner

@stuartmorgan-g stuartmorgan-g Awaiting requested review from stuartmorgan-g

@camsim99 camsim99 Awaiting requested review from camsim99

@reidbaker reidbaker Awaiting requested review from reidbaker

@LongCatIsLooong LongCatIsLooong Awaiting requested review from LongCatIsLooong

@LouiseHsu LouiseHsu Awaiting requested review from LouiseHsu LouiseHsu is a code owner

@mboetger mboetger Awaiting requested review from mboetger mboetger is a code owner

+3 more reviewers

@ezamagni-tg ezamagni-tg ezamagni-tg left review comments

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

@ash2moon ash2moon ash2moon requested changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

p: video_player platform-android platform-ios platform-macos platform-web triage-android Should be looked at in Android triage triage-ios Should be looked at in iOS triage

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@nateshmbhat @stuartmorgan-g @camsim99 @mdebbar @LongCatIsLooong @ezamagni-tg @tarrinneal @ash2moon