If you want to make an update to the project, please, create a new branch and send it for a pull request. Example:
- Switch to a new branch:
git checkout -b 'branch_name' - Do the code change
- Append the change:
git add . - Commit:
git commit -m "name of my commit" - Push to the repo and wait for a review:
git push --set-upstream origin 'branch_name'
The Stack - open-source code base and the related paper
LibGen - textbooks source
RedPajama - open-source language model
Causal language modeling - popular way of language models training by prediction the next word in a sentence
Textbooks Are All You Need - generation of textbook quality data that allowed to train a relatively small yet efficient model
LoRA - fune-tuning method by matrix decomposition
Polycoder - code generation pretrained model + dataset collection scripts
Codex - Web inteface only