Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recommendation: GemmaTokenizer requires tensorflow-text package (Python 3.12) #1274

Open
josnin opened this issue May 15, 2024 · 2 comments
Open

Comments

@josnin
Copy link

josnin commented May 15, 2024

I would like to recommend adding support for Python 3.12 to the TensorFlow Text library. As Python 3.12 is a relatively new release, it would be beneficial to ensure compatibility with this version.

I am encountering an error when using the GemmaTokenizer. The error message states:

 Exception encountered: GemmaTokenizer requires the tensorflow-text package. Please install with pip install tensorflow-text.
@davepickup
Copy link

I would also like to see tensorflow-text support Python 3.12. Tensorflow>=2.16 already supports 3.12 so it seems it would make sense to have the same compatibility in the tensorflow-text package.

@nicechester
Copy link

nicechester commented May 31, 2024

Well, I managed to compile tensorflow-text with python 3.12 on aarch64. You should be able to to that if you followed the instruction on your OS.
This is the resulting binary. https://github.com/nicechester/tensorflow-docker/blob/main/wheels/tensorflow_text-2.16.1-cp312-cp312-linux_aarch64.whl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants