Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature request] Using ArImage as InputImage parameter #875

Open
wholesomemoon opened this issue Sep 23, 2024 · 0 comments
Open

[Feature request] Using ArImage as InputImage parameter #875

wholesomemoon opened this issue Sep 23, 2024 · 0 comments
Assignees

Comments

@wholesomemoon
Copy link

wholesomemoon commented Sep 23, 2024

What's your feature request? Please describe.

In my current project, ARCore is fully implemented in native code (C/C++). The captured camera CPU buffer is of type ArImage, which is essentially same as InputImage.IMAGE_FORMAT_YUV_420_888 format. I'm providing the YUV plane data from the native layer to the Java side through JNI, and passing the camera frame to MLKit as an InputImage in Java. However, while MLKit’s does offer InputImage.IMAGE_FORMAT_YUV_420_888 along with InputImage.fromMediaImage, the image data must be provided in the form of android.media.Image.

Unfortunately, there doesn't seem to be a way to instantiate native C/C++ YUV plane buffers as android.media.Image. I currently convert the data to a format like NV21 and then pass it to MLKit using InputImage.fromByteArray with InputImage.IMAGE_FORMAT_NV21 (which works well smoothly).

However, I feel like this additional process is still no needed. If I were using ARCore in Java and utilized acquireCameraImage, the data would be provided as android.media.Image from the start, so this issue wouldn't arise. However, in my current project, ARCore Java cannot be used, so I need to use arframe_acquirecameraimage instead.

As mentioned in the issue #481, while I understand that providing MLKit as NDK would take long anyway, it would be helpful if the only InputImage could support YUV plane data directly. Or if there is another good approach that I might be unaware of and this feature addition is unnecessary, I would greatly appreciate it if you could let me know as well. Thank you. :)

Mobile environment
Android

Additional context
None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants