Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Inconsistent getPosition result #84

Open
tannker opened this issue May 7, 2024 · 3 comments
Open

Inconsistent getPosition result #84

tannker opened this issue May 7, 2024 · 3 comments

Comments

@tannker
Copy link

tannker commented May 7, 2024 •
edited
Loading

Describe the bug
After loading video, I call after few second delay videoController.positon.then((value){print(value)}. The result is inconsistent. Sometimes it returns 0, sometimes 20 or 80 milliseconds. The same issue occurs when calling position after videoController.seekTo. After seeking to the start (Duration.zero or Duration(milliseconds: 0) or to the end, the position value returned varies. The variation is in some milliseconds, but on most cases it is not equal to actual start or end.
I made some debugging and it seems that this error is already present at video_player_mdk.dart function getPosition. This function I guess already gets input from libmdk.

Additionally, I updated mdk pod from 0.25.1 to the latest 0.27 one, but the behaviour remained the same.
The test is made with latest version of fvp - 0.17.0.
I am testing on MacOS Sonoma 14.4.1.

Expected behavior
Position value is consistent across all calls.

Log

flutter: mdk.INFO: 14:30:39.909: default0x15908c470 new FrameReader...
flutter: mdk.INFO: 14:30:39.909: Registered audio backends: AudioQueue OpenAL null 
flutter: mdk.INFO: 14:30:39.909: default0x159089690 new FrameReader...
flutter: mdk.INFO: 14:30:39.909: Registered audio backends: AudioQueue OpenAL null 
flutter: mdk.INFO: 14:30:39.909: 0x1590971c0mdk::Player::Player()
flutter: mdk.INFO: 14:30:39.910: player property: video.decoder = shader_resource=0
flutter: mdk.INFO: 14:30:39.910: player property: avformat.strict = experimental
flutter: mdk.INFO: 14:30:39.910: player property: avio.protocol_whitelist = file,rtmp,http,https,tls,rtp,tcp,udp,crypto,httpproxy,data,concatf,concat,subfile
flutter: mdk.INFO: 14:30:39.910: player property: avformat.rtsp_transport = tcp
flutter: mdk.INFO: 14:30:39.910: player property: keep_open = 1
flutter: mdk.INFO: 14:30:39.910: video decoders: "VT"
flutter: mdk.INFO: 14:30:39.910: video decoders: "VT"
flutter: mdk.INFO: 14:30:39.910: default FrameReader0x15908c470 state: 0=>0=>0, 0
flutter: mdk.INFO: 14:30:39.910: default0x15908c470 virtual void mdk::FrameReader::stop(), current state: 0
flutter: mdk.INFO: 14:30:39.910: default FrameReader0x15908c470 state: 0=>0=>0, 0
flutter: mdk.INFO: 14:30:39.910: default0x15908c470 virtual void mdk::FrameReader::stop(), current state: 0
flutter: mdk.INFO: 14:30:39.910: AudioQueueStart ERROR: 0
flutter: mdk.INFO: 14:30:39.910: 0x15a047800 MediaControl.prepare(0, ...) file:///Volumes/Macintosh HD/Users//Downloads/test_videos/test.MP4
flutter: mdk.INFO: 14:30:39.910: default 0x15908c470 FrameReader.start(0, ...)
flutter: mdk.INFO: 14:30:39.910: default 0x15908c470 FrameReader::update MediaStatus 0=>0X2
flutter: fvp.FINE: 14:30:39.913: 468818744 player5788597168 onMediaStatus: MediaStatus(noMedia) => MediaStatus(+loading)
flutter: mdk.INFO: 14:30:39.914: default FrameReader0x15908c470 state: 0=>0=>1, 0
flutter: mdk.INFO: 14:30:39.914: default FrameReader0x15908c470 state: 0=>1=>1, 0
flutter: mdk.INFO: 14:30:39.914: start frame reader thread: 0x178667000
flutter: mdk.INFO: 14:30:39.914: Trying MediaIO FFmpeg
flutter: mdk.INFO: 14:30:39.914: 0x600001d1e880 open url: file:///Volumes/Macintosh HD/Users//Downloads/test_videos/test.MP4
flutter: mdk.INFO: 14:30:39.914: 0x600001d1e880 url opened
flutter: mdk.INFO: 14:30:39.914: format adts is used for file:///Volumes/Macintosh HD/Users//Downloads/test_videos/test.MP4
flutter: mdk.INFO: 14:30:39.914: default 0x15908c470 FrameReader::update MediaStatus 0X2=>0X2
flutter: mdk.INFO: 14:30:39.914: 0x15906dcd0 avio set class
flutter: mdk.INFO: 14:30:39.914: before avformat_open_input. io: 0x600001d1e880(FFmpeg/pb: 0x15906dcd0, url: file:///Volumes/Macintosh HD/Users//Downloads/test_videos/test.MP4
flutter: mdk.INFO: 14:30:39.914: after avformat_open_input. pb: 0x15906dcd0, iformat: 0x15c61dcf0 context flags: 2097280, input format flags: 67141640
flutter: mdk.INFO: 14:30:39.914: unsupported avformat options:  rtsp_transport=tcp
flutter: mdk.INFO: 14:30:39.914: Unsupported stream type: data, codec: unknown
flutter: mdk.INFO: 14:30:39.914: Unsupported stream type: data, codec: bin_data
flutter: mdk.INFO: 14:30:39.914: Format: mov,mp4,m4a,3gp,3g2,mj2, range: 0 +19136ms, bitrate: 59788566, size: 0
Metadata:
  firmware: H22.01.02.20.00
  creation_time: 2024-01-17T10:24:21.000000Z
  minor_version: 538120216
  compatible_brands: mp41
  major_brand: mp41
Streams: 4
 Video:
  stream#0, range: 0 +19130ms, frames: 1913
  codec: hevc tag: 'hvc1' profile: 1 level: 153, yuv420p, bpc:8, bpp:12(8,8,8), channels:(1,1,1), map: 0 1 2 0, bitrate: 59513249, 1920x1080, fps: 100, bframes: 0, primaries: BT709, trc: BT709, matrix: BT709, range: Full
  extra data(117): 01 01 60 00 00 00 00 00 00 00 00 00 99 F0 00 FC FD F8 F8 00 00 0F 03 A0 00 01 00 21 40 01 0C 01 FF FF 01 60 00 00 03 00 00 03 00 00 03 00 00 03 00 99 BC 0C 00 00 0F A0 00 06 1A 81 40 A1 00 01 00 27 42 01 01 01 60 00 00 03 00 00 03 00 00 03 00 00 03 00 99 A0 03 C0 80 10 E5 96 F9 24 C0 8F E5 CF 4B 01 6E 02 02 02 01 A2 00 01 00 07 44 01 E0 2D AC 09 90 
  Metadata:
   encoder: GoPro H.265 encoder
   handler_name: �GoPro H.265
   vendor_id: [0][0][0][0]
   language: eng
   timecode: 11:24:21:82
   creation_time: 2024-01-17T10:24:21.000000Z
 Audio:
  stream#1, range: 0 +19136ms, frames: 897
  codec: aac tag: 'mp4a' profile: 1 level: -99, sample size: 0/24, block align: 0, frame_size: 1024, f32p stereo(2) @48000Hz, bitrate: 189255
  extra data(5): 11 90 00 00 00 
  Metadata:
   handler_name: �GoPro AAC  
   vendor_id: [0][0][0][0]
   language: eng
   timecode: 11:24:21:82
   creation_time: 2024-01-17T10:24:21.000000Z

flutter: mdk.INFO: 14:30:39.914: default FrameReader0x15908c470 request to pause 1, loaded: 0.
flutter: mdk.INFO: 14:30:39.914: default FrameReader0x15908c470 state: 0=>1=>2, 0
flutter: mdk.INFO: 14:30:39.914: default FrameReader0x15908c470 state requested: 2, current: 0
flutter: mdk.INFO: 14:30:39.915: virtual void mdk::AudioBackendAudioQueue::pause(bool)138 ERROR>>> AudioQueuePause(queue_): -50
flutter: mdk.INFO: 14:30:39.915: 
flutter: mdk.INFO: 14:30:39.915: 0x15a047800 prepared callback is invoked
flutter: fvp.FINE: 14:30:39.925: 468818744 player5788597168 onEvent: metadata -  - 0
flutter: mdk.INFO: 14:30:39.925: default 0x15908c470 FrameReader::update MediaStatus 0X2=>0X4
flutter: fvp.FINE: 14:30:39.925: 468818744 player5788597168 onMediaStatus: MediaStatus(+loading) => MediaStatus(+loaded)
flutter: mdk.INFO: 14:30:39.926: ***buffering progress 0%***
flutter: mdk.INFO: 14:30:39.926: default 0x15908c470 FrameReader::update MediaStatus 0X4=>0X14
flutter: mdk.INFO: 14:30:39.926: virtual void mdk::AudioBackendAudioQueue::pause(bool)138 ERROR>>> AudioQueuePause(queue_): -50
flutter: mdk.INFO: 14:30:39.926: 
flutter: fvp.FINE: 14:30:39.926: 468818744 player5788597168 onMediaStatus: MediaStatus(+loaded) => MediaStatus(+loaded+buffering)
flutter: mdk.INFO: 14:30:39.926: ++++++++++++BUFFERING START++++++++++++
flutter: mdk.INFO: 14:30:39.926: 
flutter: fvp.FINE: 14:30:39.926: 468818744 player5788597168 onEvent: reader.buffering -  - 0
flutter: fvp.FINE: 14:30:39.927: 468818744 player5788597168 onEvent: metadata -  - 0
flutter: mdk.INFO: 14:30:39.928: default FrameReader0x15908c470 state: 0=>2=>1, 1
flutter: mdk.INFO: 14:30:39.928: default FrameReader0x15908c470 update state: 0=>1
flutter: fvp.FINE: 14:30:39.928: 468818744 player5788597168 onPlaybackStateChanged: PlaybackState.stopped => PlaybackState.playing
flutter: mdk.INFO: 14:30:39.928: default FrameReader0x15908c470 state: 1=>1=>2, 1
flutter: mdk.INFO: 14:30:39.928: default FrameReader0x15908c470 update state: 1=>2
flutter: fvp.FINE: 14:30:39.928: 468818744 player5788597168 onPlaybackStateChanged: PlaybackState.playing => PlaybackState.paused
flutter: mdk.INFO: 14:30:39.928: starting decode loop thread mdk.vdec0@0x17877f000
flutter: fvp.FINE: 14:30:39.928: 468818744 player5788597168 onEvent: thread.video -  - 1
flutter: mdk.INFO: 14:30:39.928: video stream#0 starting decoding loop from decoder index 0...
flutter: mdk.INFO: 14:30:39.928: creating video decoder: VT...
flutter: mdk.INFO: 14:30:39.928: 0x159243058mdk::NativeVideoBufferPool::NativeVideoBufferPool()
flutter: mdk.INFO: 14:30:39.928: Try to use IOSurface OpenGL(ES) interop: true
flutter: mdk.INFO: 14:30:39.928: opening video decoder: VT...
flutter: mdk.INFO: 14:30:39.928: starting decode loop thread mdk.adec1@0x17880b000
flutter: fvp.FINE: 14:30:39.928: 468818744 player5788597168 onEvent: thread.audio -  - 1
flutter: mdk.INFO: 14:30:39.929: audio stream#1 starting decoding loop from decoder index 0...
flutter: mdk.INFO: 14:30:39.929: creating audio decoder: auto...
flutter: mdk.INFO: 14:30:39.929: opening audio decoder: FFmpeg...
flutter: mdk.INFO: 14:30:39.929: opening ffmpeg audio decoder: aac ...
flutter: mdk.INFO: 14:30:39.929: AVCodec.Audio[aac.] decoder: 1 Single threads
flutter: mdk.INFO: 14:30:39.929: aac f32p, stereo @48000Hz
flutter: fvp.FINE: 14:30:39.929: 468818744 player5788597168 onEvent: decoder.audio - FFmpeg - 0
flutter: mdk.INFO: 14:30:39.929: ***buffering progress 100%***
flutter: mdk.INFO: 14:30:39.929: default 0x15908c470 FrameReader::update MediaStatus 0X14=>0X124
flutter: mdk.INFO: 14:30:39.929: virtual void mdk::AudioBackendAudioQueue::pause(bool)138 ERROR>>> AudioQueuePause(queue_): -50
flutter: mdk.INFO: 14:30:39.929: 
flutter: fvp.FINE: 14:30:39.929: 468818744 player5788597168 onMediaStatus: MediaStatus(+loaded+buffering) => MediaStatus(+loaded+prepared+buffered)
flutter: mdk.INFO: 14:30:39.929: ++++++++++++BUFFERING END++++++++++++
flutter: mdk.INFO: 14:30:39.929: 
flutter: fvp.FINE: 14:30:39.929: 468818744 player5788597168 onEvent: reader.buffering -  - 100
flutter: mdk.INFO: 14:30:39.929: audio stream#1 sending 1 invalid AOT frame @0.000000s. seeking: 0
flutter: mdk.INFO: 14:30:39.929: 0x15a047800 1st audio frame @0.000000
flutter: mdk.INFO: 14:30:39.929: 0x15a047800 seek end audio frame @0.000000 seek_pos_: -1, sync_ao_ 1
flutter: mdk.INFO: 14:30:39.929: AudioRenderer format: f32, stereo @48000Hz, requested: f32p, stereo @48000Hz
flutter: mdk.INFO: 14:30:39.929: AudioRenderer format changed and reinitialize
flutter: mdk.INFO: 14:30:39.929: codec 'hvc1' for VT
flutter: mdk.INFO: 14:30:39.929: 0x1590971d0 setRenderAPI(3, 0x0)
flutter: mdk.INFO: 14:30:39.929: getVideoOutContext 0x0 => 0x60000317b030. vo count: 1
flutter: mdk.INFO: 14:30:39.929: 0x1590971d0 setVideoSurfaceSize(1920x1080, 0x0)
flutter: mdk.INFO: 14:30:40.041: VT hardware decode support: 1
flutter: mdk.INFO: 14:30:40.041: NALU length bytes: 4
flutter: mdk.INFO: 14:30:40.041: <CMVideoFormatDescription 0x600003ca8f90 [0x1f29b48c0]> {
	mediaType:'vide' 
	mediaSubType:'hvc1' 
	mediaSpecific: {
		codecType: 'hvc1'		dimensions: 1920 x 1080 
	} 
	extensions: {{
    CVImageBufferColorPrimaries = "ITU_R_709_2";
    CVImageBufferTransferFunction = "ITU_R_709_2";
    CVImageBufferYCbCrMatrix = "ITU_R_709_2";
    Depth = 24;
    FormatName = hvc1;
    FullRangeVideo = 1;
    SampleDescriptionExtensionAtoms =     {
        hvcC = {length = 117, bytes = 0x01016000 00000000 00000000 99f000fc ... 074401e0 2dac0990 };
    };
}}
}
flutter: mdk.INFO: 14:30:40.041: setLoop(0), now 0/0
flutter: mdk.INFO: 14:30:40.041: 0x15a047800 virtual void mdk::MediaControlPush::setState(PlaybackState)@1051 requested state 2=>2, current state 2. status: 0X124
flutter: mdk.INFO: 14:30:40.041: UsingHardwareAcceleratedVideoDecoder: 1
flutter: fvp.FINE: 14:30:40.041: 468818744 player5788597168 onEvent: decoder.video - VT - 0
flutter: mdk.INFO: 14:30:40.042: decode stored recovery packets: 0
flutter: mdk.INFO: 14:30:40.042: video stream#0 sending 1 invalid AOT frame @0.000000s. seeking: 0
flutter: mdk.INFO: 14:30:40.042: 0x15a047800 1st video frame to render @0.000000s, sync time: -0.001000
flutter: mdk.INFO: 14:30:40.042: VideoRenderer clear buffered frames
flutter: mdk.INFO: 14:30:40.042: 0-track seek end video frame @0.000000 seek_pos_: -1
flutter: mdk.INFO: 14:30:40.042: 0x15a047800 1st video frame to render @0.000000s, sync time: -0.001000
flutter: fvp.FINE: 14:30:40.042: 468818744 player5788597168 onEvent: video - size - 0
flutter: mdk.INFO: 14:30:40.042: VideoRenderer WxH=1920.000000x1080.000000 1920.000000x1080.000000, frame: 1920x1080
flutter: mdk.INFO: 14:30:40.042: update transform 2d. scale: (1.000000, 1.000000), orientation: 0
flutter: mdk.INFO: 14:30:40.042: m: mat<4, 4>(
flutter: mdk.INFO: 14:30:40.042: 1  0  0  0  
flutter: mdk.INFO: 14:30:40.042: 0  1  0  0  
flutter: mdk.INFO: 14:30:40.042: 0  0  1  0  
flutter: mdk.INFO: 14:30:40.042: 0  0  0  1  
flutter: mdk.INFO: 14:30:40.042: )
flutter: mdk.INFO: 14:30:40.042: unknown, bpc:0, bpp:0(), channels:(), map: 0 0 0 0 => mapped nv12, bpc:8, bpp:12(8,16), channels:(1,2), map: 0 1 2 0
flutter: mdk.INFO: 14:30:40.042: Metal pixel format 0: MTLPixelFormatR8Unorm
flutter: mdk.INFO: 14:30:40.042: Metal pixel format 1: MTLPixelFormatRG8Unorm
flutter: mdk.INFO: 14:30:40.042: frame color: BT709 BT709 BT709 Full. gamma: 2.200000, signal peak: 1.000000
flutter: mdk.INFO: 14:30:40.042:  - 
flutter: mdk.INFO: 14:30:40.042: output color: BT709 UNSPECIFIED RGB Full. gamma: 2.200000, signal peak: 1.000000
flutter: mdk.INFO: 14:30:40.042: rgb primaries matrix: mat<3, 3>(
flutter: mdk.INFO: 14:30:40.042: 1  0  0  
flutter: mdk.INFO: 14:30:40.042: 0  1  0  
flutter: mdk.INFO: 14:30:40.042: 0  0  1  
flutter: mdk.INFO: 14:30:40.043: )
flutter: mdk.INFO: 14:30:40.043: scale: 1.000000, color transform: mat<4, 4>(
flutter: mdk.INFO: 14:30:40.043: 1  0  1.5748  -0.7874  
flutter: mdk.INFO: 14:30:40.043: 1  -0.187324  -0.468124  0.327724  
flutter: mdk.INFO: 14:30:40.043: 1  1.8556  0  -0.9278  
flutter: mdk.INFO: 14:30:40.043: 0  0  0  1  
flutter: mdk.INFO: 14:30:40.043: )
flutter: mdk.INFO: 14:30:40.043: EQ: mat<4, 4>(
flutter: mdk.INFO: 14:30:40.043: 1  0  0  0  
flutter: mdk.INFO: 14:30:40.043: 0  1  0  0  
flutter: mdk.INFO: 14:30:40.043: 0  0  1  0  
flutter: mdk.INFO: 14:30:40.043: 0  0  0  1  
flutter: mdk.INFO: 14:30:40.043: )
flutter: mdk.INFO: 14:30:40.043: shader id: 2321
flutter: mdk.INFO: 14:30:40.043: dump UBO layout:
align @offset +size /stride address name ~ path: type -> parent
16@0 +64/16 0x60000104daa8 u_colorMatrix ~ u_colorMatrix: mat4 -> 0x0            

flutter: fvp.FINE: 14:30:40.043: 468818744 player5788597168 onEvent: render.video - 1st_frame - 0
flutter: mdk.INFO: 14:30:40.043: Using audio backend: AudioQueue
flutter: mdk.INFO: 14:30:40.043: default FrameReader0x15908c470 request to pause 1, loaded: 4.
flutter: mdk.INFO: 14:30:40.043: default FrameReader0x15908c470 state: 2=>2=>2, 0
flutter: mdk.INFO: 14:30:40.043: 0x15a047800 ao opened: 1, reopen: 1, bad ao: 0
flutter: mdk.INFO: 14:30:40.043: 0x6000002268e8 filter graph: 
flutter: mdk.INFO: 14:30:40.043: +-----------+
|    src    |default--[48000Hz fltp:stereo]--swr:default
| (abuffer) |
+-----------+

                                          +---------------+
swr:default--[48000Hz flt:stereo]--default|      dst      |
                                          | (abuffersink) |
                                          +---------------+

                                           +-------------+
src:default--[48000Hz fltp:stereo]--default|     swr     |default--[48000Hz flt:stereo]--dst:default
                                           | (aresample) |
                                           +-------------+
@wang-bin
Copy link
Owner

wang-bin commented May 8, 2024

If video is paused after load, the position should be the first video frame timestamp(there is a bug in current code and will be fixed soon). If not paused, the position varies. seek is the same.

@tannker
Copy link
Author

tannker commented May 8, 2024

Ok. Yes, what I have noticed is the position after initial load (sometimes 0, 20, 80 ms). I also noticed the same issue with seekTo to either to the start (Duration.zero - again 0, 20, 80 ms) or to the end (videoController.value.duration - position value differs slightly from the actual duration).

@wang-bin
Copy link
Owner

upgrade libmdk

pod cache clean mdk
pod deintegrate ios/Runner.xcodeproj
rm ios/Podfile.lock
rm -rf ios/Pods

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wang-bin @tannker