AI Models

Built-In LLMs

Various different LLMs are supported by ESBMC-AI.

OpenAI

• gpt-3.5-turbo
• gpt-3.5-16k
• gpt-4
• gpt-4-32k

Technology Innovation Institute (TII)

• falcon-7b: (Prototype/WIP) Uses text-generation-inference on the backend.

Custom LLM

ESBMC-AI has support of custom AI. The config supports adding custom AI models that are text-generation-inference from Hugging Face. These are specified in config.json inside the ai_custom field. The ai_model field selects which AI model to use. If the AI model is chosen and does not exist built in, then the list inside ai_custom will be checked. This means that all the entries inside ai_custom must be unique and not match any of the built-in first class AI. When adding a custom_ai entry. The name of the AI will be the entry name. The entry takes three fields, max_tokens, url, and config_message:

• The max_tokens are the acceptable max tokens that the AI can accept.
• The url is the text-generation-inference server URL that the LLM is hosted in.
• The config_message is a custom configuration message that will wrap the messages, this is used to convert text generation LLM into Chat LLMs.
• All the values mentioned are dependent on the LLM that is implemented.

There are tokens that are replaced inside the config_message in order to insert the conversation:

• {history}: The previous messages sent by the AI model.
• {user_prompt}: The last message of the user to the AI.

Example

Here is an example entry for the ai_custom field in the config.json:

"example-llm": {
  "max_tokens": 1024,
  "url": "www.example.com",
  "config_message": "{history}\n{user_prompt}\n"
}