Multiple model selection with remote service #1049
Conversation
…ort multiple models chatqna with wrapper is required
…ong with remote tgi/tei service with instructions
…ital Assistant, Code Generator, Content Summarizer respectively
…ntent exceeds the window height
…and Faqgen response
… adjust to different screen sizes
…n the chatqna prompt section
…different models from dropdown
for more information, see https://pre-commit.ci
| http_proxy: ${http_proxy} | ||
| https_proxy: ${https_proxy} | ||
| TEI_EMBEDDING_ENDPOINT: ${TEI_EMBEDDING_ENDPOINT} | ||
| LANGCHAIN_API_KEY: ${LANGCHAIN_API_KEY} |
There was a problem hiding this comment.
langchain related environment variables are not used anymore, please remove.
There was a problem hiding this comment.
Thanks for the review. Removed langchain related environment variables
…docker compose files Signed-off-by: sgurunat <[email protected]>
for more information, see https://pre-commit.ci
| @@ -0,0 +1,57 @@ | |||
| # Copyright (C) 2024 Intel Corporation | |||
There was a problem hiding this comment.
Do we need different files to support this feature? I think it's better to integrate them into one file and get environment variables automatically.
There was a problem hiding this comment.
I created a separate file because in the actual file (to run all services locally) we use a variable called {host_ip} which will point to local machine host ip and different port is configured for different services. But in case of remote host there might be a different endpoints/ports for each service i.e for embedding different endpoint, for llm different endpoint etc.. Because of which I created a separate file with required environment variables so that there is no confusion for users to run it locally or remote. Please let me know if you still feel to integrate it?
There was a problem hiding this comment.
Thanks for your explanation. I still think it should be integrated into one file, and explain the ${host_ip}/${remote_host_ip} stuff in the README.md file. Because for the remote scenario, users should be able to set any service as a remote service, not only tgi/tei embedding.
Further more, please refer to this pr. The environment variables for docker compose have been replaced with service name, and each examples should follow this schema for users to deploy services easier.
Thank you.
Description
This PR contains changes related to multiple models selection in ProductivitySuite ChatQnA along with some minor enhancements to the UI. Also it contains docker compose files and instructions to run ProductivitySuite on Intel Gaudi server with remote TGI/TEI services.
Type of change
New feature (non-breaking change which adds new functionality)
Others (enhancement, documentation, validation, etc.)
New Features:
Add chatqna_wrapper.py along with updated Dockerfile.wrapper. To support multiple models chatqna with wrapper is required
ProductivitySuite: Add docker compose files for Intel Gaudi server along with remote tgi/tei service with instructions
ProductivitySuite UI: Add multiple models support. Choose different models from dropdown
Enhancements:
ProductivitySuite UI: Update names of ChatQnA, CodeGen, DocSum to Digital Assistant, Code Generator, Content Summarizer respectively
ProductivitySuite UI: Update Docsum to have vertical scroll bar if content exceeds the window height
ProductivitySuite UI: Remove <|eot_id|> string from the Chat, Docsum and Faqgen response
ProductivitySuite UI: Update contextWrapper and contextTitle width to adjust to different screen sizes
ProductivitySuite UI: Show system prompt input field always to edit in the chatqna prompt section
ProductivitySuite UI: Update max_new_tokens into max_tokens