[DeepVision port] Adding CLIP + pretrained weights #1947
Conversation
| ) | ||
|
|
||
|
|
||
| class CLIPTokenizer(object): |
There was a problem hiding this comment.
I think this if no where else is an excellent opportunity to import from KerasNLP. We already have a BPE tokenizer and downstream GPT2 tokenizer!
There was a problem hiding this comment.
Agreed! I'll rely on those then :)
Thanks for the heads up!
|
@jbischof you got in a bit early on the review 😅 Good idea on importing portions from KNLP! |
|
Actually one complexity on borrowing the NLP tokenizer is that it is written in TF and cannot be ported to Keras Core due to tf_text dependency. So maybe you can borrow the Stable Diffusion one instead. |
|
@DavidLandup0 Thank you for this PR! are you still working on this PR? I will create a feature branch for adding the CLIP model and you can add your changes to that - https://github.com/keras-team/keras-cv/tree/CLIP |
|
@DavidLandup0 I have cherry picked your changes to this branch - https://github.com/keras-team/keras-cv/tree/CLIP |
What does this PR do?
As discussed in #1933 - setting up a draft PR for porting CLIP and associated layers into KCV. Draft PR for now with placeholder main model dump, layers and tests incoming soon. Will tag once ready for review.
Before submitting
Pull Request section?
to it if that's the case. Porting DeepVision into KerasCV #1933
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@ianstenbit just tagging so you can follow the progress as it comes in. Otherwise, no need to spend time until it's un-drafted for review :)