Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to continue pre-training Bloom? #366

Open
ShinoharaHare opened this issue Feb 26, 2023 · 2 comments
Open

How to continue pre-training Bloom? #366

ShinoharaHare opened this issue Feb 26, 2023 · 2 comments

Comments

@ShinoharaHare
Copy link

ShinoharaHare commented Feb 26, 2023 •
edited
Loading

Hi
I'm trying to continue pre-training the bloom-560m on my own dataset on a single GPU.
I modified this script to fit my case.
However, i cannot figure out how to load the checkpoint.

Is there any guide for what i'm doing?
@lwmlyy
Copy link

lwmlyy commented Mar 22, 2023

Hi,
did you find any solution on this?

@noob-ctrl
Copy link

@ShinoharaHare Hi, have you solved this problem？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lwmlyy @ShinoharaHare @noob-ctrl