Deep Cody: skip query rewrite for search tool #6082
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change adds a
skipQueryRewrite
option to theContextRetriever.retrieveContext
method, which is used when retrieving context for the Search tool.This allows the Deep Cody to bypass the query rewriting step that makes an extra LLM request when using the search tool. The reason behind this is because Deep Cody has already return a proper search query that doesn't need rewritten into keyword.
The
ContextRetriever._retrieveContext
method has been updated to handle theskipQueryRewrite
option, and theSearchTool
class now passestrue
for this option when callingretrieveContext
.Minor prompt improvement (formatting and wording).
I've also notice some response quality regression after switching to use 3.5 Haiku for the reviewing step, where Haiku was not able to fetch the right context resulting in the wrong answer. Switching back to 3.5 Haiku:
Test plan
After: took
8797.651249999995ms
to complete 2 review loopsExample 2: " what models are available for dotcom users defined in the codebase?"
completion time:
6397.538499999995ms
Before
review loop took
14535.338709000032ms
to complete 1 review loopsExample 2: " what models are available for dotcom users defined in the codebase?"
completion time:
10159.913833000232ms
Changelog