Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add prompt field in RAGTool #130

Closed
wants to merge 3 commits into from

Conversation

SuZhou-Joe
Copy link
Member

@SuZhou-Joe SuZhou-Joe commented Jan 16, 2024

Description

[Describe what this change achieves]

Issues Resolved

#132

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

codecov bot commented Jan 16, 2024

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (7e4c8d5) 80.83% compared to head (23a2502) 80.85%.

Files Patch % Lines
.../main/java/org/opensearch/agent/tools/RAGTool.java 83.33% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main     #130      +/-   ##
============================================
+ Coverage     80.83%   80.85%   +0.01%     
- Complexity      195      196       +1     
============================================
  Files            13       13              
  Lines          1002     1008       +6     
  Branches        133      134       +1     
============================================
+ Hits            810      815       +5     
  Misses          141      141              
- Partials         51       52       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@xinyual
Copy link
Collaborator

xinyual commented Jan 16, 2024

So will we have a default prompt for RAG tool?

@SuZhou-Joe
Copy link
Member Author

#132

@mingshl
Copy link
Contributor

mingshl commented Jan 16, 2024

So will we have a default prompt for RAG tool?

if the prompt is going to add in when register a tool, we should either check if empty prompt then throw exception, or give the prompt a default.

Copy link
Contributor

@mingshl mingshl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (prompt.trim().isEmpty() || prompt == null) {
     throw new IllegalArgumentException("prompt cannot be empty");
}

@zhichao-aws
Copy link
Member

Can we set a default prompt for claude model? like #125

@SuZhou-Joe
Copy link
Member Author

So will we have a default prompt for RAG tool?

I once heard that we should never put prompt in our code, it should be a config from user. Will add a validation in Tool.

@SuZhou-Joe
Copy link
Member Author

Based on discussion with @mingshl , we decide to add a validation on prompt field instead of adding a default prompt as the prompt may contain claude-specific template like: Human: turn, which will introduce confusion here.
@mingshl @xinyual @zhichao-aws Could you please help to review and get it merged?

@mingshl
Copy link
Contributor

mingshl commented Jan 17, 2024

Hi @SuZhou-Joe ,

IMO, it's better the the prompt should be carried through the tool id, not the RAGTool interface. so I took a look at MLModelTool which is similar to RAGTool that takes a prompt when register the tool and ask llm model a question in a request, MLModelTool doesn't need the prompt in the constructor, the prompt should added through LLMSpec and then in every MLPredictionTaskRequest, it will carry the prompt to use in model prediction.

based on the previous testing on ml-commons feature branch, the RAGToold without the prompt in the constructor was working, I would like to double check if the setting of the RAGTool might lead to issue #132

@xinyual
Copy link
Collaborator

xinyual commented Jan 18, 2024

So will we have a default prompt for RAG tool?

if the prompt is going to add in when register a tool, we should either check if empty prompt then throw exception, or give the prompt a default.

Yes. I think we could add a default prompt if the prompt is empty like #125

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants