[BUG] Correct response but hallucinated questions/answers included

[rmuhawieh]

Pre-check

• I have searched the existing issues and none cover this bug.

Description

90% of the time, I get the response I'm looking for. But, sometimes, when getting a response back, it will give the correct response but then include its own questions and answers within the response. Sometimes it will also loop back to trying to answer a question asked a few questions ago despite it having nothing to do with the current question. The format of the response is usually like the below. I noticed it seems to happen most often for a question that requires a short, straightforward response. Does this require a small tweak to the system prompt? Is anyone else facing this as well?

Correct answer here...

Answer...

or other times

Correct answer here...

[In response to the question you asked earlier, ....]

Steps to Reproduce

There's no reproducing it. The above explains it the best I can

Expected Behavior

Respond to the question

Actual Behavior

Response to the question with additional hallucinations beneath it

Environment

Linux

Additional Information

No response

Version

No response

Setup Checklist

• Confirm that you have followed the installation instructions in the project’s documentation.
• Check that you are using the latest version of the project.
• Verify disk space availability for model storage and data processing.
• Ensure that you have the necessary permissions to run the project.

NVIDIA GPU Setup Checklist

• Check that the all CUDA dependencies are installed and are compatible with your GPU (refer to CUDA's documentation)
• Ensure an NVIDIA GPU is installed and recognized by the system (run nvidia-smi to verify).
• Ensure proper permissions are set for accessing GPU resources.
• Docker users - Verify that the NVIDIA Container Toolkit is configured correctly (e.g. run sudo docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi)