-
Notifications
You must be signed in to change notification settings - Fork 2
Contributing
Here, we will add notes that will help you to make meaningful contributions to factgenie.
If you want to get a quick feedback or actively participate in development of factgenie, join our public Slack workspace:
We hope that you find factgenie useful! With your feedback, we can make it even better.
Please, report any bugs or enhancement ideas to Github issues with appropriate tags.
You can help us immensely by:
- using factgenie for your experiments,
- recommending factgenie to your friends and colleagues,
- sharing factgenie on social media.
If you use factgenie for your experiments, please cite our INLG 2024 System Demonstrations paper.
We are developing factgenie in a small team of people along with many other ongoing projects. Sadly, that means that we can put only a limited amount of energy into ongoing factgenie development.
If you are a developer that can help us with making factgenie better, you are thus welcome aboard!
The simplest way you can help is by adding external resources: datasets, model outputs, and annotations that are available in factgenie for download.
These resources show users what is possible in factgenie and provide a stepping stone for their own projects. In some cases, the users may use your resources for their own experiments directly.
π‘ Please, read the Data Management page first to understand the concepts mentioned here.
To add a resource, you need to:
- Write a custom dataset loader.
- On top of the required methods, also implement the
download()
method that downloads the dataset (optionally along with model outputs and annotations) from an external source.- See
loaders/dataset.py
and existing loaders for more details.
- See
- Add your dataset to
datasets.yml
. - Make a pull request (see below).
We welcome any pull requests solving the open issues in our issue tracker!
If you wish to integrate your change into factgenie, please fork the repository and make a pull request.
Here is a step-by-step guide:
- Fork the repository: Click the "Fork" button at the top right of the repository page.
-
Clone your fork:
git clone https://github.com/your-username/factgenie.git
-
Create a branch:
git checkout -b your-branch-name
- Make your changes: Implement your feature or bug fix.
-
Commit your changes:
git commit -m "Description of your changes"
-
Push to your fork:
git push origin your-branch-name
- Create a Pull Request: Go to the original repository and click "New Pull Request".
We will try to review and merge your pull request as soon as possible.
We use black for code formatting. You can install it with pip install black
and run it with black .
in the root directory of the repository.
Black can be also integrated into your IDE. For example, in VS Code, you can install the Black formatter extension and set the following configuration in your settings.json
:
{
"python.formatting.provider": "black",
"editor.formatOnSave": true,
}
Note that we are using the line length of 120 characters, as noted in pyproject.toml
:
[tool.black]
line-length = 120
Please, do not to commit any large files directly in the factgenie repository: use git lfs
instead.
We currently use git lfs
for storing the demo video of the framework.
Links to git lfs
documentation:
- π§ Setup
- ποΈ Data Management
- π€ LLM Annotations
- π₯ Crowdsourcing Annotations
- βοΈ Generating Outputs
- π Analyzing Annotations
- π» Command Line Interface
- π± Contributing
- π Importing a custom dataset
- π¬ Generating outputs
- π Customizing data visualization
- π€ Annotating outputs with an LLM
- π¨βπΌ Annotating outputs with human annotators