Can I fine-tune without captions? #2645
Unanswered
brainsbriansbrains
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I was trying my first fine-tune a few days ago, I'm typically just doing loras but now want to take a step up in the world. I have a lot of photos and didn't want to caption all of them--but when I started training, it errored out on "no meta_cap.json file detected" and such errors. I found out this only went away when I added captions and enabled training settings like generate metadata.
Of course, I could just remove this error-flagging in the code, but I'm not sure how necessary it is. Can we not fine-tune without captions to just teach the model new visual concepts and bias it's output to the domain of the fine-tuning data? Are captions necessary? Is there a more elegant way to get around this flagging while fine-tuning than just adding the same basic caption to each photo? Would appreciate some input from people who have done fine-tuning before. Thanks in advance
Beta Was this translation helpful? Give feedback.
All reactions