Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: refactor/standardise model providers code + add "get provider key" #251

Merged
merged 10 commits into from
Nov 14, 2024

Conversation

wonderwhy-er
Copy link
Collaborator

@wonderwhy-er wonderwhy-er commented Nov 11, 2024

Goal

Goal was to provide link to get API key for first time users
To achieve that I standardised provider list to include name, static and dynamic model lists and API key url and label.
For LM Studio link and label are about downloading LM Studio, for Ollama too

PR Summary

  1. Constants Update:

    • Introduces ProviderInfo and PROVIDER_LIST, organizing providers with models and API key links.
    • Refactors model fetching to be dynamic for supported providers.
  2. Base Chat Component Update:

    • Uses PROVIDER_LIST dynamically for provider selection, removing hardcoded options.
  3. API Key Management Component Update:

    • Added link to get API keys if providers has the link
  4. Added Dynamic Model list for OpenRouter

  5. Made server use API keys from cookies instead of request body - there were issues with them not being actual in request body

  6. Fixed Google API keys provider - API key was passed as argument while options with it as property was expected

@wonderwhy-er wonderwhy-er marked this pull request as draft November 11, 2024 21:49
@wonderwhy-er wonderwhy-er changed the title feat: Refactor/standardise model providers code + add "get provider key" feat: refactor/standardise model providers code + add "get provider key" Nov 12, 2024
@wonderwhy-er wonderwhy-er marked this pull request as ready for review November 12, 2024 09:26
# Conflicts:
#	app/components/chat/BaseChat.tsx
@wonderwhy-er
Copy link
Collaborator Author

I merged in dynamic model list for OpenRouter into this PR. Also fixed bunch of things. Added list to description.

Would be good to merge.

@chrismahoney
Copy link
Collaborator

chrismahoney commented Nov 14, 2024

Edit: Moving to a review comment

@chrismahoney chrismahoney self-requested a review November 14, 2024 03:36
Copy link
Collaborator

@chrismahoney chrismahoney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wonderwhy-er I'm hitting the same wall that the build is hitting here regarding p.getDynamicModels() being called potentially undefined. Could you take a look and see if you get this as well merged with current main? This was the only issue and after changes below it worked great.

image

constants.ts:237 is the initializeModelList function, feel free to DM if you want to get together and check it out. The typecheck isn't what needs to pass, I just get a similar error on the current branch.

Here's a diff I used to get it running and see my Ollama models working correctly with some changes in tsconfig - hopefully a couple of hints here

diff --git a/app/components/chat/Chat.client.tsx b/app/components/chat/Chat.client.tsx
index 2407e86..3029dc2 100644
--- a/app/components/chat/Chat.client.tsx
+++ b/app/components/chat/Chat.client.tsx
@@ -269,8 +269,8 @@ export const ChatImpl = memo(({ initialMessages, storeMessageHistory }: ChatProp
             scrollTextArea();
           },
           model,
-          provider,
-          apiKeys
+          provider?.name,
+          apiKeys,
         );
       }}
     />
diff --git a/app/utils/constants.ts b/app/utils/constants.ts
index 32bc816..75d0307 100644
--- a/app/utils/constants.ts
+++ b/app/utils/constants.ts
@@ -8,7 +8,7 @@ export const PROVIDER_REGEX = /\[Provider: (.*?)\]\n\n/;
 export const DEFAULT_MODEL = 'claude-3-5-sonnet-latest';
 
 
-export type ProviderInfo = {
+type ProviderInfo = {
   staticModels: ModelInfo[],
   name: string,
   getDynamicModels?: () => Promise<ModelInfo[]>,
@@ -234,8 +234,8 @@ async function getLMStudioModels(): Promise<ModelInfo[]> {
 
 async function initializeModelList(): Promise<ModelInfo[]> {
   MODEL_LIST = [...(await Promise.all(
-    PROVIDER_LIST.filter(p => !!p.getDynamicModels).map(p => p.getDynamicModels()))).flat(), ...staticModels];
+    PROVIDER_LIST.filter(p => !!p.getDynamicModels).map(p => p.getDynamicModels && p.getDynamicModels()))).flat(), ...staticModels];
   return MODEL_LIST;
 }
 
-export { getOllamaModels, getOpenAILikeModels, getLMStudioModels, initializeModelList, getOpenRouterModels, PROVIDER_LIST };
+export { getOllamaModels, getOpenAILikeModels, getLMStudioModels, initializeModelList, getOpenRouterModels, PROVIDER_LIST, ProviderInfo };
diff --git a/tsconfig.json b/tsconfig.json
index fd161f9..4b704a9 100644
--- a/tsconfig.json
+++ b/tsconfig.json
@@ -2,7 +2,7 @@
   "compilerOptions": {
     "lib": ["DOM", "DOM.Iterable", "ESNext"],
     "types": ["@remix-run/cloudflare", "vite/client", "@cloudflare/workers-types/2023-07-01"],
-    "isolatedModules": true,
+    "isolatedModules": false,
     "esModuleInterop": true,
     "jsx": "react-jsx",
     "module": "ESNext",
@@ -12,7 +12,7 @@
     "strict": true,
     "allowJs": true,
     "skipLibCheck": true,
-    "verbatimModuleSyntax": true,
+    "verbatimModuleSyntax": false,
     "forceConsistentCasingInFileNames": true,
     "baseUrl": ".",
     "paths": {

@chrismahoney
Copy link
Collaborator

Created #276 to capture any additional issues/PRs that should adapt into this refactor after the merge.

@wonderwhy-er
Copy link
Collaborator Author

Pushed in fix for type check error

@wonderwhy-er
Copy link
Collaborator Author

Added additional type fix for ProviderInfo import.
Should work with dev command now too.

@chrismahoney chrismahoney merged commit 1e6f8ef into coleam00:main Nov 14, 2024
1 check passed
@chrismahoney
Copy link
Collaborator

Alright! Looks good based on your changes @wonderwhy-er

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants