5. Hosting on AI Horde

If you have spare compute, you are encouraged to contribute to the AI Horde by hosting a model of your choice. This initiative is a community-driven, open-source project that enables volunteers to host large language models, making them accessible to all at no cost.

To get started, install Aphrodite Engine and launch a server with the Kobold endpoint.

1. Clone the AI Horde Worker repo.

git clone https://github.com/Haidra-Org/AI-Horde-Worker

2. Create the bridgeData.yaml file

A template is already provided. Simply rename the bridgeData_template.yaml to bridgeData.yaml and modify a few things:

• api_key: Your horde API key. Register here to acquire one.
• max_threads: specifies how many concurrent requests your worker should run. Higher values require more VRAM.
• scribe_name: your custom worker name.
• kai_url: the Aphrodite URL. By default, this should be http://localhost:2242.
• max_length: this specifies the max number of tokens every request can make. A good value is 512.
• max_context_length: The maximum context length of the horde worker. Set this to your model's default max length, or whatever value you passed to --max-model-len when launching the engine.

That should be all you need to edit for now - don't touch the other options unless you know what you're doing.

3. Launch the Horde worker

Simply run the provided bridge script:

sh ./horde-scribe-bridge.sh

This should create a brand new conda environment and install all the requirements, then launch the bridge.

Thanks for contributing to the horde!