Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New extension: AudioVisualizer #1408

Open
3 tasks done
giannpls opened this issue Oct 12, 2024 · 12 comments
Open
3 tasks done

New extension: AudioVisualizer #1408

giannpls opened this issue Oct 12, 2024 · 12 comments
Labels
✨ New extension A new extension

Comments

@giannpls
Copy link

giannpls commented Oct 12, 2024

Description

This extension provides the ability to record audio from the microphone and play it back through the speakers. It also provides the ability to monitor real-time audio data from the microphone and master output, which can be used to create audio visualizations such as a spectrogram, oscilloscope, RMS/Peak meter, or other animations which respond to properties of the audio signal.

How to use the extension

The extension provides events (StartMicRecording/StopMicRecording) to record audio from the microphone using the MediaStream Recording API, and then the SaveMicRecording event pushes those recordings to the project data so that the recording can be played back through the sound manager with the PlayMicRecording event. It also provides access to real-time audio data from the microphone and the Howler master output with GetAnalyzerFrequencyData/GetAnalyzerTimeDomainData events (which use WebAudio AnalyserNode objects), to allow a user to create audio visualizations

Checklist

  • I've followed all of the best practices.
  • I confirm that this extension can be integrated to this GitHub repository, distributed and MIT licensed.
  • I am aware that the extension may be updated by anyone, and do not need my explicit consent to do so.

What tier of review do you aim for your extension?

Community (Unreviewed)

Example file

42db3c5d-d779-4c9f-b95e-b81f0f802bd7.zip

Extension file

AudioVisualizer.zip

@giannpls giannpls added the ✨ New extension A new extension label Oct 12, 2024
Copy link
Contributor

👋 Thanks for your submission! Unfortunately, all submitted extensions have to meet some minimal quality standard - the extension best practices - and our system has found that your extension does not fully comply 😔. You can find all of those rules on the GDevelop wiki.

The following issues have been found by the system concerning the respect of those best practices by your extension:

  ⟶ ❌ [Dots in sentences]: Field 'shortDescription' of the extension description misses a dot at the end of the sentence!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'GetAnalyzerFrequencyData' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerFrequencyData' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'sentence' of the function 'GetAnalyzerFrequencyData' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerFrequencyData parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerFrequencyData parameter 'Format' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerFrequencyData parameter 'OutArray' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerFrequencyData parameter 'OutSize' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'GetAnalyzerTimeDomainData' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerTimeDomainData' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'sentence' of the function 'GetAnalyzerTimeDomainData' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerTimeDomainData parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerTimeDomainData parameter 'Format' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerTimeDomainData parameter 'OutArray' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerTimeDomainData parameter 'OutSize' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'SetAnalyzerFFTSize' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'SetAnalyzerFFTSize' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'sentence' of the function 'SetAnalyzerFFTSize' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'SetAnalyzerFFTSize parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'SetAnalyzerFFTSize parameter 'Size' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'AnalyzerFrequencyBinCount' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'AnalyzerFrequencyBinCount' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'AnalyzerFrequencyBinCount parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'FrequencyAtFFTBin' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FrequencyAtFFTBin' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FrequencyAtFFTBin parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FrequencyAtFFTBin parameter 'Bin' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'FFTBinAtFrequency' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FFTBinAtFrequency' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FFTBinAtFrequency parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FFTBinAtFrequency parameter 'Frequency' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'SampleRate' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'SampleRate' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'StartMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'StartMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'sentence' of the function 'StartMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'PauseMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'PauseMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'sentence' of the function 'PauseMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'ResumeMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'ResumeMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'sentence' of the function 'ResumeMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'StopMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'StopMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'sentence' of the function 'StopMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'SaveMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'SaveMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'sentence' of the function 'SaveMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'SaveMicRecording parameter 'Name' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'PlayMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'PlayMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'sentence' of the function 'PlayMicRecording' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'PlayMicRecording parameter 'Name' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'PlayMicRecording parameter 'Volume' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'MicRecorderState' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'MicRecorderState' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'RecordingExists' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'RecordingExists' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'sentence' of the function 'RecordingExists' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'RecordingExists parameter 'Name' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'fullName' of the function 'MicRecorderIsInState' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'MicRecorderIsInState' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'sentence' of the function 'MicRecorderIsInState' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'MicRecorderIsInState parameter 'State' is not filled out!

Please update your original submission post with a new zip file containing your extensions updated to follow those guidelines 🙏

@EyadMahm0ud
Copy link

I loved it, however you can't leave a function description or sentence blank or it doesn't meet the best practices.

@giannpls
Copy link
Author

giannpls commented Oct 12, 2024 via email

Copy link
Contributor

👋 Thanks for your submission! Unfortunately, all submitted extensions have to meet some minimal quality standard - the extension best practices - and our system has found that your extension does not fully comply 😔. You can find all of those rules on the GDevelop wiki.

The following issues have been found by the system concerning the respect of those best practices by your extension:

  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'GetAnalyzerFrequencyData' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'GetAnalyzerTimeDomainData' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'SetAnalyzerFFTSize' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'AnalyzerFFTSize' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'AnalyzerFrequencyBinCount' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'AnalyzerSampleRate' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'FrequencyAtFFTBin' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'FFTBinAtFrequency' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'StartMicRecording' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'PauseMicRecording' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'ResumeMicRecording' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'StopMicRecording' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'SaveMicRecording' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'PlayMicRecording' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'MicRecorderState' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'RecordingExists' misses a dot at the end of the sentence!
  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'MicRecorderIsInState' misses a dot at the end of the sentence!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerFrequencyData parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerFrequencyData parameter 'Format' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerFrequencyData parameter 'OutArray' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerFrequencyData parameter 'OutSize' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerTimeDomainData parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerTimeDomainData parameter 'Format' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerTimeDomainData parameter 'OutArray' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'GetAnalyzerTimeDomainData parameter 'OutSize' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'SetAnalyzerFFTSize parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'SetAnalyzerFFTSize parameter 'Size' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'AnalyzerFFTSize parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'AnalyzerFrequencyBinCount parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'AnalyzerSampleRate parameter 'Node' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FrequencyAtFFTBin parameter 'Bin' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FrequencyAtFFTBin parameter 'FFTSize' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FrequencyAtFFTBin parameter 'SampleRate' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FFTBinAtFrequency parameter 'Frequency' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FFTBinAtFrequency parameter 'FFTSize' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'FFTBinAtFrequency parameter 'SampleRate' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'SaveMicRecording parameter 'Name' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'PlayMicRecording parameter 'Name' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'PlayMicRecording parameter 'Volume' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'RecordingExists parameter 'Name' is not filled out!
  ⟶ ❌ [Filled out names and descriptions]: Required field 'description' of the function 'MicRecorderIsInState parameter 'State' is not filled out!

Please update your original submission post with a new zip file containing your extensions updated to follow those guidelines 🙏

@EyadMahm0ud
Copy link

EyadMahm0ud commented Oct 13, 2024

You must add period in the end of all descriptions

@giannpls
Copy link
Author

You must add period in the end of all descriptions

I resubmitted again, I think this time it was accepted

Copy link
Contributor

👋 Thanks for your submission! Unfortunately, all submitted extensions have to meet some minimal quality standard - the extension best practices - and our system has found that your extension does not fully comply 😔. You can find all of those rules on the GDevelop wiki.

The following issues have been found by the system concerning the respect of those best practices by your extension:

  ⟶ ❌ [Dots in sentences]: Field 'description' of the function 'PlayMicRecordingOnChannel' misses a dot at the end of the sentence!

Please update your original submission post with a new zip file containing your extensions updated to follow those guidelines 🙏

@kubanskakommuna
Copy link

Hello! I'd like to use your extension, but a one bug spoils my desire to use it. When I recorded an audio and changed a scene, scene doesn't open. Please, fix it.

@giannpls
Copy link
Author

giannpls commented Oct 20, 2024

Hello! I'd like to use your extension, but a one bug spoils my desire to use it. When I recorded an audio and changed a scene, scene doesn't open. Please, fix it.

I think I am able to reproduce this. Do you need to be able to access the recording in the new scene?

EDIT: storing the recording in project data and performing hot reload seemed to cause the issue, and I couldn't find a way to fix it. For now I changed the save mechanism to bypass project data and send the audio to the sound manager directly. It should resolve the issue with changing scenes. Please retry with the latest attachment.

Note that I did not test if you will be able to access the recording in the new scene. If that's what you need, it may require some additional work and may require storing the recording in project data.

@giannpls
Copy link
Author

Note that I did not test if you will be able to access the recording in the new scene. If that's what you need, it may require some additional work and may require storing the recording in project data.

Updated again so recordings can persist through scene changes if desired. Added buttons to switch between mic recording and audio visualization scenes. So far it works for me on PC and iPhone

@kubanskakommuna
Copy link

I hope you will port this plugin to Android, that would be great.

@giannpls
Copy link
Author

I hope you will port this plugin to Android, that would be great.

I don't have an android device for myself, but I tested with my wife's phone and it seems to work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✨ New extension A new extension
Projects
None yet
Development

No branches or pull requests

3 participants