Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ollama: Allow specifying a bearer token #19493

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

kov
Copy link

@kov kov commented Oct 20, 2024

Closes #19491

Release Notes:

  • Added `token_bearer' setting for the ollama provider, to allow for authenticated API endpoints

Copy link

cla-bot bot commented Oct 20, 2024

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Gustavo Noronha Silva.
This is most likely caused by a git client misconfiguration; please make sure to:

  1. check if your git client is configured with an email to sign commits git config --list | grep email
  2. If not, set it up using git config --global user.email [email protected]
  3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

@cla-bot cla-bot bot added the cla-signed The user has signed the Contributor License Agreement label Oct 20, 2024
@notpeter
Copy link
Member

Hi, thanks for this PR!

The bearer token shouldn't be stored in settings as it's a secret. We support reading API keys from the environment (e.g. ANTHROPIC_API_KEY, OPENAI_API_KEY) or keeping credentials in the platform specific keychain (search for cx.read_credentials and cx.write_credentials).

For example:

fn authenticate(&self, cx: &mut ModelContext<Self>) -> Task<Result<()>> {
if self.is_authenticated() {
Task::ready(Ok(()))
} else {
let api_url = AllLanguageModelSettings::get_global(cx)
.anthropic
.api_url
.clone();
cx.spawn(|this, mut cx| async move {
let (api_key, from_env) = if let Ok(api_key) = std::env::var(ANTHROPIC_API_KEY_VAR)
{
(api_key, true)
} else {
let (_, api_key) = cx
.update(|cx| cx.read_credentials(&api_url))?
.await?
.ok_or_else(|| anyhow!("credentials not found"))?;
(String::from_utf8(api_key)?, false)
};

Unlike the other providers, this key would be optional (e.g. local ollama requires no auth), but we'd want to support reading OLLAMA_API_KEY from the environment or providing a key on the Assistant configuration page like we do for OpenAI/Anthropic/Gemini.
Screenshot 2024-10-20 at 12 52 07

)

Release Notes:

- Added OLLAMA_API_KEY and UI to set up key for the ollama provider to allow
  for authenticated API endpoints
@kov kov marked this pull request as draft October 21, 2024 00:06
@kov
Copy link
Author

kov commented Oct 21, 2024

That makes sense, thank you for the quick feedback! I have started on that path. It is quite a bit more code. I got the UI to render, most of the code structure is in place, I believe, but it is still not working and the UI could use a little bit more love. I will try to get it finished during the week, may need to wait for the weekend, though.

@maxdeviant maxdeviant changed the title ollama: allow specifying a bearer token (#19491) ollama: Allow specifying a bearer token Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed The user has signed the Contributor License Agreement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow authentication for ollama and other custom providers
2 participants