Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Modify audio codec API to use AudioBuffer #335

Merged
merged 10 commits into from
Sep 18, 2024

Conversation

yezhan10
Copy link
Contributor

@yezhan10 yezhan10 commented Sep 3, 2024

What's included in this PR:

  1. All audio encoders consume AudioBuffer, return AudioFrame pointer.
  2. All audio decoders consume AudioFrame, return AudioBuffer shared pointer obj. Since it's possible that a audio frame has multiple decoded PCM data. So split the original Decode() function to Decode() and GetDecodedAudioFrame().
  3. Added unit tests for AudioPipe, OpusEncoder.

Timestamp improvement will be included in next PR.

CMakeLists.txt Outdated Show resolved Hide resolved
CMakeLists.txt Show resolved Hide resolved
include/AudioPipe.h Outdated Show resolved Hide resolved
include/AudioPipe.h Outdated Show resolved Hide resolved
include/audio.h Outdated Show resolved Hide resolved
src/AudioEncoderWorker.cpp Outdated Show resolved Hide resolved
include/AudioPipe.h Outdated Show resolved Hide resolved
include/AudioPipe.h Outdated Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
@murillo128
Copy link
Member

In order to be able to progress this PR, can we split it in two PRs?

The first one about method changes to use the AudioBuffer and the second one to fix the timestamp calculation inside the AudioPipe?

@yezhan10
Copy link
Contributor Author

yezhan10 commented Sep 9, 2024

In order to be able to progress this PR, can we split it in two PRs?

The first one about method changes to use the AudioBuffer and the second one to fix the timestamp calculation inside the AudioPipe?

@murillo128 @bcostdolby I have updated the implementation to use a queue of audio buffers in audio pipe, and also added timestamp calculation for audio buffer based on decoder pts. if we all like this approach better, i will clean up the code a bit in my next commit. This version still has Encode(const AudioBuffer::const_shared& audioBuffer, BYTE* out, int outLen), I will update this to return an encoded frame in next PR most likely. Also, maybe we can use audiobuffer pool as an improvement in the next PR as well to avoid frequent audio buffer creation.

include/AudioBuffer.h Outdated Show resolved Hide resolved
include/AudioBuffer.h Outdated Show resolved Hide resolved
include/AudioBuffer.h Outdated Show resolved Hide resolved
include/AudioBuffer.h Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
src/AudioPipe.cpp Outdated Show resolved Hide resolved
@yezhan10 yezhan10 changed the title improve audio encoder timestamp Modify audio codec API to use AudioBuffer Sep 11, 2024
@yezhan10 yezhan10 marked this pull request as ready for review September 11, 2024 20:15
@yezhan10 yezhan10 force-pushed the dev/yezhan/DIOS-6699 branch 2 times, most recently from f889cf5 to 7bdc6ac Compare September 11, 2024 21:31
@murillo128 murillo128 changed the title Modify audio codec API to use AudioBuffer refactor: Modify audio codec API to use AudioBuffer Sep 12, 2024
Copy link
Member

@murillo128 murillo128 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking better, still some changes I would like to make

include/AudioBuffer.h Outdated Show resolved Hide resolved
include/AudioBuffer.h Outdated Show resolved Hide resolved
include/AudioBuffer.h Outdated Show resolved Hide resolved
include/AudioBuffer.h Outdated Show resolved Hide resolved
include/AudioBuffer.h Outdated Show resolved Hide resolved
include/AudioPipe.h Outdated Show resolved Hide resolved
src/opus/OpusEncoder.h Outdated Show resolved Hide resolved
include/audio.h Outdated Show resolved Hide resolved
src/gsm/gsmcodec.cpp Outdated Show resolved Hide resolved
src/gsm/gsmcodec.cpp Outdated Show resolved Hide resolved
src/gsm/gsmcodec.cpp Outdated Show resolved Hide resolved
Copy link
Member

@murillo128 murillo128 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

src/g711/pcmacodec.cpp Outdated Show resolved Hide resolved
@murillo128 murillo128 merged commit ef65fdb into master Sep 18, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants