French answers

[jmbernabotto]

Hello, I'm a French user. I would like the answer of my question is French and not English ! Which file I have to transform to do that ? Or, which model I have to take ?

[imartinez]

Take a look a the comments in #135 . Same apply for french. Let us know if you need more specific info about how to change the prompt!

[skaterjesus]

Actually found a way to set the RetrievalQA chain language.
My example is for german, but can be easily adapted for other languages

...

prompt_template = """
    Verwende die folgenden Kontextinformationen, um die am Ende gestellte Frage zu beantworten. Wenn du die Antwort nicht weißt, sag einfach, dass du sie nicht weißt, versuche nicht, eine Antwort zu erfinden.

    {context}

    Frage: {question}
    Antwort auf Deutsch:
"""
PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)
chain_type_kwargs = {"prompt": PROMPT}

qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents=True, chain_type_kwargs=chain_type_kwargs)

... 

[jmbernabotto]

I don't understand where is this input !
Could you please be more explicit ?

[KeulzZz]

I feel like it could be in the privategpt.py instead of:

    qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents= not args.hide_source)

haven't tried yet

[sime2408]

Putting {question} inside prompt using gpt4all model didn't work for me so I removed that part. But then answers are not so great.
I think the better solution would be to use T5 encoder decoder models from Google which are suitable for this like google/flan-t5-xxl, but I am not sure which model is trained for chat there. I am referring to google translate models. Anyone experienced with this?

https://huggingface.co/docs/transformers/model_doc/t5

from transformers import T5ForConditionalGeneration, 
T5Tokenizer
import gradio as grad

text2text_tkn= T5Tokenizer.from_pretrained("t5-small") 
mdl = T5ForConditionalGeneration.from_pretrained("t5-small") 

def text2text_translation(text):
  inp = " translate English to German:: "+text 
  enc = text2text_tkn(inp, return_tensors="pt") 
  tokens = mdl.generate(**enc)
  response=text2text_tkn.batch_decode(tokens)
  return response


para=grad.Textbox(lines=1, label="English Text", placeholder="Text in English")

out=grad.Textbox(lines=1, label="German Translation")

grad.Interface(text2text_translation, inputs=para, 
outputs=out).launch()


# requirements.txt
# gradio
# transformers
# torch
# transformers[sentencepiece]

Related to that I remember using Helsinki-NLP/opus-mt-en-de once, there is Helsinki-NLP/opus-mt-en-fr too...

[amroeser]

I have adapted the code from @skaterjesus , so it seems to work for the German language. The name 'PromptTemplate' was not defined correctly at first.
This is how the privateGPT.py file should look for German:

from dotenv import load_dotenv
from langchain.chains import RetrievalQA
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.vectorstores import Chroma
from langchain.llms import GPT4All, LlamaCpp
from langchain.prompts import PromptTemplate
import os
import argparse

load_dotenv()

embeddings_model_name = os.environ.get("EMBEDDINGS_MODEL_NAME")
persist_directory = os.environ.get('PERSIST_DIRECTORY')

model_type = os.environ.get('MODEL_TYPE')
model_path = os.environ.get('MODEL_PATH')
model_n_ctx = os.environ.get('MODEL_N_CTX')
target_source_chunks = int(os.environ.get('TARGET_SOURCE_CHUNKS', 4))

from constants import CHROMA_SETTINGS

def main():
    # Parse the command line arguments
    args = parse_arguments()
    embeddings = HuggingFaceEmbeddings(model_name=embeddings_model_name)
    db = Chroma(persist_directory=persist_directory, embedding_function=embeddings, client_settings=CHROMA_SETTINGS)
    retriever = db.as_retriever(search_kwargs={"k": target_source_chunks})
    # activate/deactivate the streaming StdOut callback for LLMs
    callbacks = [] if args.mute_stream else [StreamingStdOutCallbackHandler()]
    # Prepare the LLM
    match model_type:
        case "LlamaCpp":
            llm = LlamaCpp(model_path=model_path, n_ctx=model_n_ctx, callbacks=callbacks, verbose=False)
        case "GPT4All":
            llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', callbacks=callbacks, verbose=False)
        case _default:
            print(f"Model {model_type} not supported!")
            exit
    prompt_template = """
        Verwende die folgenden Kontextinformationen, um die am Ende gestellte Frage zu beantworten. Wenn du die Antwort nicht weißt, sag einfach, dass du sie nicht weißt, versuche nicht, eine Antwort zu erfinden.

        {context}

        Frage: {question}
        Antwort auf Deutsch:
    """
    PROMPT = PromptTemplate(
        template=prompt_template, input_variables=["context", "question"]
    )
    chain_type_kwargs = {"prompt": PROMPT}
    qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents=True, chain_type_kwargs=chain_type_kwargs)
    # Interactive questions and answers
    while True:
        query = input("\nGeben Sie eine Frage ein: ")
        if query == "exit":
            break

        # Get the answer from the chain
        res = qa(query)
        answer, docs = res['result'], [] if args.hide_source else res['source_documents']

        # Print the result
        print("\n\n> Frage:")
        print(query)
        print("\n> Antwort:")
        print(answer)

        # Print the relevant sources used for the answer
        for document in docs:
            print("\n> " + document.metadata["source"] + ":")
            print(document.page_content)

def parse_arguments():
    parser = argparse.ArgumentParser(description='privateGPT: Ask questions to your documents without an internet connection, '
                                                 'using the power of LLMs.')
    parser.add_argument("--hide-source", "-S", action='store_true',
                        help='Use this flag to disable printing of source documents used for answers.')

    parser.add_argument("--mute-stream", "-M",
                        action='store_true',
                        help='Use this flag to disable the streaming StdOut callback for LLMs.')

    return parser.parse_args()


if __name__ == "__main__":
    main()

For French, simply adapt the following part:

# Laden der Umgebungsvariablen
load_dotenv()

# Setzen Sie die Umgebungsvariable für das Modell für Französisch
os.environ["EMBEDDINGS_MODEL_NAME"] = "camembert-base"

# Abrufen des Modellnamens aus den Umgebungsvariablen
embeddings_model_name = os.environ.get("EMBEDDINGS_MODEL_NAME")


# ...

prompt_template = """
    Utilisez les informations de contexte suivantes pour répondre à la question posée à la fin. Si vous ne connaissez pas la réponse, dites simplement que vous ne la connaissez pas, n'inventez pas de réponse.

    {context}

    Question : {question}
    Réponse en français :
"""