-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: refactor/standardise model providers code + add "get provider key" #251
Conversation
…e who have it for first time users
# Conflicts: # app/components/chat/BaseChat.tsx
I merged in dynamic model list for OpenRouter into this PR. Also fixed bunch of things. Added list to description. Would be good to merge. |
Edit: Moving to a review comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@wonderwhy-er I'm hitting the same wall that the build is hitting here regarding p.getDynamicModels() being called potentially undefined. Could you take a look and see if you get this as well merged with current main? This was the only issue and after changes below it worked great.
constants.ts:237 is the initializeModelList
function, feel free to DM if you want to get together and check it out. The typecheck isn't what needs to pass, I just get a similar error on the current branch.
Here's a diff I used to get it running and see my Ollama models working correctly with some changes in tsconfig - hopefully a couple of hints here
diff --git a/app/components/chat/Chat.client.tsx b/app/components/chat/Chat.client.tsx
index 2407e86..3029dc2 100644
--- a/app/components/chat/Chat.client.tsx
+++ b/app/components/chat/Chat.client.tsx
@@ -269,8 +269,8 @@ export const ChatImpl = memo(({ initialMessages, storeMessageHistory }: ChatProp
scrollTextArea();
},
model,
- provider,
- apiKeys
+ provider?.name,
+ apiKeys,
);
}}
/>
diff --git a/app/utils/constants.ts b/app/utils/constants.ts
index 32bc816..75d0307 100644
--- a/app/utils/constants.ts
+++ b/app/utils/constants.ts
@@ -8,7 +8,7 @@ export const PROVIDER_REGEX = /\[Provider: (.*?)\]\n\n/;
export const DEFAULT_MODEL = 'claude-3-5-sonnet-latest';
-export type ProviderInfo = {
+type ProviderInfo = {
staticModels: ModelInfo[],
name: string,
getDynamicModels?: () => Promise<ModelInfo[]>,
@@ -234,8 +234,8 @@ async function getLMStudioModels(): Promise<ModelInfo[]> {
async function initializeModelList(): Promise<ModelInfo[]> {
MODEL_LIST = [...(await Promise.all(
- PROVIDER_LIST.filter(p => !!p.getDynamicModels).map(p => p.getDynamicModels()))).flat(), ...staticModels];
+ PROVIDER_LIST.filter(p => !!p.getDynamicModels).map(p => p.getDynamicModels && p.getDynamicModels()))).flat(), ...staticModels];
return MODEL_LIST;
}
-export { getOllamaModels, getOpenAILikeModels, getLMStudioModels, initializeModelList, getOpenRouterModels, PROVIDER_LIST };
+export { getOllamaModels, getOpenAILikeModels, getLMStudioModels, initializeModelList, getOpenRouterModels, PROVIDER_LIST, ProviderInfo };
diff --git a/tsconfig.json b/tsconfig.json
index fd161f9..4b704a9 100644
--- a/tsconfig.json
+++ b/tsconfig.json
@@ -2,7 +2,7 @@
"compilerOptions": {
"lib": ["DOM", "DOM.Iterable", "ESNext"],
"types": ["@remix-run/cloudflare", "vite/client", "@cloudflare/workers-types/2023-07-01"],
- "isolatedModules": true,
+ "isolatedModules": false,
"esModuleInterop": true,
"jsx": "react-jsx",
"module": "ESNext",
@@ -12,7 +12,7 @@
"strict": true,
"allowJs": true,
"skipLibCheck": true,
- "verbatimModuleSyntax": true,
+ "verbatimModuleSyntax": false,
"forceConsistentCasingInFileNames": true,
"baseUrl": ".",
"paths": {
Created #276 to capture any additional issues/PRs that should adapt into this refactor after the merge. |
Pushed in fix for type check error |
Added additional type fix for ProviderInfo import. |
Alright! Looks good based on your changes @wonderwhy-er |
Goal
Goal was to provide link to get API key for first time users
To achieve that I standardised provider list to include name, static and dynamic model lists and API key url and label.
For LM Studio link and label are about downloading LM Studio, for Ollama too
PR Summary
Constants Update:
ProviderInfo
andPROVIDER_LIST
, organizing providers with models and API key links.Base Chat Component Update:
PROVIDER_LIST
dynamically for provider selection, removing hardcoded options.API Key Management Component Update:
Added Dynamic Model list for OpenRouter
Made server use API keys from cookies instead of request body - there were issues with them not being actual in request body
Fixed Google API keys provider - API key was passed as argument while options with it as property was expected