Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance prompts to ask the LLM to plan things ahead #62

Open
tobitege opened this issue Sep 23, 2024 · 5 comments
Open

Enhance prompts to ask the LLM to plan things ahead #62

tobitege opened this issue Sep 23, 2024 · 5 comments

Comments

@tobitege
Copy link

I find it helpful to include in my initial prompt to first ask the LLM to make a detailed plan about the task and then follow its steps.

In this context, it'd be to create steps for whatever is set out in an issue's description.

Some rejection/exclusion criteria could minimize the LLM run away with hours of work without an unreachable goal.
In case of rejection, the LLM could be allowed to post a GitHub comment for that issue with its reasoning that lead to rejection.

It first needs to e.g. recognize whether the issue is actually about the repo itself and not about e.g. ideas how other software is doing something or to integrate a totally different software that would be in the "huge effort" category (in our understanding).

In addition, have it write the plan to an actual text file (with issue number in it), so it can reference it throughout the process ("Let's deal with step 6...").
In case of failure (or hard restrictions/limits), it could be asked to resume on it later.

TL;DR tell the LLM to not go wild, but make a plan first.

@codegefluester
Copy link

Did you try adding a .openhands_instructions file in the root folder of your repository? That file appears to be read and injected into the initial prompt.

@tobitege
Copy link
Author

That file is only used by the openhands resolver package and was only recently added (2 weeks ago).
It is not being read by OH or its agents, last I checked

@codegefluester
Copy link

That file is only used by the openhands resolver package and was only recently added (2 weeks ago). It is not being read by OH or its agents, last I checked

I ran a resolver GitHub action just today and it did read the file + injected it into the task description for the agent, verified by looking at the logs.

@tobitege
Copy link
Author

Yes, that's what I wrote: only the resolver uses it, not if you use OH itself.

@codegefluester
Copy link

Yes, that's what I wrote: only the resolver uses it, not if you use OH itself.

Ah sorry, didn't parse that correctly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants