Skip to content

Commit

Permalink
Align README.md with docs/ and minor fixes / improvements (#214)
Browse files Browse the repository at this point in the history
* Remove `license_header.txt` in favour of `LICENSE`

* Remove `.cache/**/*`

* Update `.gitignore` from `Python.gitignore`

See https://github.com/github/gitignore/blob/main/Python.gitignore

* Update `description` in `index.md` to match `README`'s

* Update `README.md`
  • Loading branch information
alvarobartt authored Jan 5, 2024
1 parent 854e33e commit 3160a09
Show file tree
Hide file tree
Showing 56 changed files with 103 additions and 99 deletions.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Diff not rendered.
Binary file removed .cache/plugin/social/Roboto-Black.ttf
Binary file not shown.
Binary file removed .cache/plugin/social/Roboto-BlackItalic.ttf
Binary file not shown.
Binary file removed .cache/plugin/social/Roboto-Bold.ttf
Binary file not shown.
Binary file removed .cache/plugin/social/Roboto-BoldItalic.ttf
Binary file not shown.
Binary file removed .cache/plugin/social/Roboto-Italic.ttf
Binary file not shown.
Binary file removed .cache/plugin/social/Roboto-Light.ttf
Binary file not shown.
Binary file removed .cache/plugin/social/Roboto-LightItalic.ttf
Binary file not shown.
Binary file removed .cache/plugin/social/Roboto-Medium.ttf
Binary file not shown.
Binary file removed .cache/plugin/social/Roboto-MediumItalic.ttf
Binary file not shown.
Binary file removed .cache/plugin/social/Roboto-Regular.ttf
Binary file not shown.
Binary file removed .cache/plugin/social/Roboto-Thin.ttf
Binary file not shown.
Binary file removed .cache/plugin/social/Roboto-ThinItalic.ttf
Binary file not shown.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
77 changes: 59 additions & 18 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,37 +3,78 @@ __pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
dist/
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/

# Virtual environments
venv/
env/
ENV/
.venv/
.ENV/
.installed.cfg
*.egg
MANIFEST

# IDEs and editors
.idea/
.vscode/
*.sublime-project
*.sublime-workspace

# Test files
.pytest_cache/
coverage.xml
# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Sphinx documentation
docs/_build/

# Jupyter Notebook
.ipynb_checkpoints

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
.python-version

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# mkdocs documentation
/site

# Other
*.log
*.swp
.DS_Store
site/

# Jupyter Notebook
*.ipynb_checkpoints
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ repos:
exclude: ^docs/snippets/
args:
- --license-filepath
- license_header.txt
- LICENSE
- --fuzzy-match-generates-todo
# - --remove-header

Expand Down
108 changes: 42 additions & 66 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,122 +1,98 @@
<div align="center">
<h1>⚗️ distilabel</h1>
<p><em>AI Feedback (AIF) framework for building datasets and labellers with LLMs</em></p>
<p><em>AI Feedback (AIF) framework for building datasets with and for LLMs.</em></p>
</div>

![overview](https://github.com/argilla-io/distilabel/assets/36760800/360110da-809d-4e24-a29b-1a1a8bc4f9b7)

> [!TIP]
> To discuss, get support, or give feedback [join Argilla's Slack Community](https://join.slack.com/t/rubrixworkspace/shared_invite/zt-whigkyjn-a3IUJLD7gDbTZ0rKlvcJ5g) and you will be able to engage with our amazing community and also with the core developers of `argilla` and `distilabel`.
## What's `distilabel`?

`distilabel` is a framework for AI engineers to align LLMs using RLHF-related methods (e.g. reward models, DPO).

The initial focus is LLM fine-tuning and adaptation but we'll be extending it for predictive NLP use cases soon.
![overview](https://github.com/argilla-io/distilabel/assets/36760800/360110da-809d-4e24-a29b-1a1a8bc4f9b7)

Main use cases are:
## Features

1. As an AI engineer I want to **build domain-specific instruction datasets** to fine-tune OSS LLMs with increased accuracy.
2. As an AI engineer I want to **build domain-specific and diverse preference datasets** to use RLHF-related methods and align LLMs (e.g, increase the ability to follow instructions or give truthful responses).
- Integrations with the most popular libraries and APIs for LLMs: HF Transformers, OpenAI, vLLM, etc.
- Multiple tasks for Self-Instruct, Preference datasets and more.
- Dataset export to Argilla for easy data exploration and further annotation.

> [!WARNING]
> `distilabel` is currently under active development and we're iterating quickly, so take into account that we may introduce breaking changes in the releases during the upcoming weeks, and also the `README` might be outdated the best place to get started is the [documentation](http://distilabel.argilla.io/).
## Motivation

🔥 Recent projects like [Zephyr](https://huggingface.co/collections/HuggingFaceH4/zephyr-7b-6538c6d6d5ddd1cbb1744a66) and [Tulu](https://huggingface.co/collections/allenai/tulu-v2-suite-6551b56e743e6349aab45101) have shown it's possible to **build powerful open-source models with DPO and AI Feedback** (AIF) datasets.

👩‍🔬 There's a lot of exciting research in the AIF space, such as [UltraFeedback](https://huggingface.co/datasets/openbmb/UltraFeedback) (the dataset leveraged by Zephyr and Tulu), [JudgeLM](https://github.com/baaivision/JudgeLM), or [Prometheus](https://huggingface.co/kaist-ai/prometheus-13b-v1.0).

🚀 However, going beyond research efforts and applying AIF at scale it's different. For enterprise and production use, we need framework that implements **key AIF methods on a robust, efficient and scalable way**. This framework should enable AI engineers to build custom datasets at scale for their own use cases.

👩‍🎓 This, combined with humans-in-the-loop for improving dataset quality is the next big leap for OSS LLM models.

⚗️ `distilabel` aims to bridge this gap.

## Key features
## Installation

* 🤖 **Leverage OSS models and APIs**: 🤗 transformers, OpenAI, 🤗 Inference Endpoints, vLLM, llama.cpp, and more to come.
```sh
pip install distilabel --upgrade
```

* 💻 **Scalable and extensible**: Scalable implementations of existing methods (e.g. UltraFeedback). Easily extensible to build and configure your own labellers.
Requires Python 3.8+

* 🧑‍🦱 **Human-in-the-loop**: One line of code integration with Argilla to improve and correct datasets.
In addition, the following extras are available:

## Quickstart
- `hf-transformers`: for using models available in [transformers](https://github.com/huggingface/transformers) package via the `TransformersLLM` integration.
- `hf-inference-endpoints`: for using the [HuggingFace Inference Endpoints](https://huggingface.co/inference-endpoints) via the `InferenceEndpointsLLM` integration.
- `openai`: for using OpenAI API models via the `OpenAILLM` integration.
- `vllm`: for using [vllm](https://github.com/vllm-project/vllm) serving engine via the `vLLM` integration.
- `llama-cpp`: for using [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) as Python bindings for `llama.cpp`.
- `argilla`: for exporting the generated datasets to [Argilla](https://argilla.io/).

### Installation
## Example

Install with `pip` (requires Python 3.8+):
To run the following example you must install `distilabel` with both `openai` and `argilla` extras:

```bash
pip install distilabel[openai,argilla]
```sh
pip install "distilabel[openai,argilla]" --upgrade
```

### Try it out

After installing, you can immediately start experimenting with `distilabel`:

- **Explore Locally**: Follow the example below to build a preference dataset for DPO/RLHF.
- **Interactive Notebook**: Prefer an interactive experience? Try our Google Colab Notebook!

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1rO1-OlLFPBC0KPuXQOeMpZOeajiwNoMy?usp=sharing)

### Example: Build a preference dataset for DPO/RLHF
Then run the following example:

```python
from datasets import load_dataset
from distilabel.llm import OpenAILLM
from distilabel.pipeline import pipeline
from distilabel.tasks import TextGenerationTask

# Load a dataset with instructions from the Hub
dataset = (
load_dataset("HuggingFaceH4/instruction-dataset", split="test[:5]")
load_dataset("HuggingFaceH4/instruction-dataset", split="test[:10]")
.remove_columns(["completion", "meta"])
.rename_column("prompt", "input")
)

# Use `OpenAILLM` (running `gpt-3.5-turbo`) to generate responses for given inputs
generator = OpenAILLM(
task=TextGenerationTask(),
max_new_tokens=512,
# openai_api_key="sk-...",
)
# Create a `Task` for generating text given an instruction.
task = TextGenerationTask()

# Create a `LLM` for generating text using the `Task` created in
# the first step. As the `LLM` will generate text, it will be a `generator`.
generator = OpenAILLM(task=task, max_new_tokens=512)

# Create a pre-defined `Pipeline` using the `pipeline` function and the
# `generator` created in step 2. The `pipeline` function will create a
# `labeller` LLM using `OpenAILLM` with the `UltraFeedback` task for
# instruction following assessment.
pipeline = pipeline("preference", "instruction-following", generator=generator)

# Build a preference dataset comparing two responses focused on the instruction-following skill of the LLM
dataset = pipeline.generate(dataset)
```

The resulting dataset can already be used for preference tuning (a larger version of it). But beware these AIF dataset are imperfect. To get the most out of AIF, push to Argilla for human feedback:
Additionally, you can push the generated dataset to Argilla for further exploration and annotation:

```python
import argilla as rg

rg.init(
api_key="<YOUR_ARGILLA_API_KEY>",
api_url="<YOUR_ARGILLA_API_URL>"
)
rg.init(api_url="<YOUR_ARGILLA_API_URL>", api_key="<YOUR_ARGILLA_API_KEY>")

# Convert the dataset to Argilla format
rg_dataset = dataset.to_argilla()

# Push the dataset to Argilla
rg_dataset.push_to_argilla(name="preference-dataset", workspace="admin")
```

https://github.com/argilla-io/distilabel/assets/1107111/be34c95c-8be4-46ef-9437-cbd2a7687e30

### More examples
## More examples

Find more examples of different use cases of `distilabel` under [`examples/`](./examples/).

## Roadmap
Or check out the following Google Colab Notebook:

- [x] Add Critique Models and support for Prometheus OSS
- [x] Add a generator with multiple models
- [ ] Train OSS labellers to replace OpenAI labellers
- [ ] Add labellers to evolve instructions generated with self-instruct
- [ ] Add labellers for predictive NLP tasks: text classification, information extraction, etc.
- [ ] Open an issue to suggest a feature!
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1rO1-OlLFPBC0KPuXQOeMpZOeajiwNoMy?usp=sharing)

## Contribute

Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
description: Distilabel is an AI Feedback (AIF) framework to build datasets with and for LLMs.
description: Distilabel is an AI Feedback (AIF) framework for building datasets with and for LLMs.
---
# distilabel

Expand Down
13 changes: 0 additions & 13 deletions license_header.txt

This file was deleted.

0 comments on commit 3160a09

Please sign in to comment.