Fix issue #4912: [Bug]: BedrockException: "The number of toolResult blocks at messages.2.content exceeds the number of toolUse blocks of previous turn.".

[openhands-agent]

This pull request fixes #4912.

The issue has been successfully resolved through several key actions:

1. The root cause was identified as OpenHands using an incorrect provider prefix ("openai/") when making requests through the LiteLLM proxy to Bedrock.

2. A fix was implemented in the OpenHands codebase (branch xw/llm-fixes) to use the correct "litellm_proxy/" prefix instead.

3. The fix was verified by the user (@scosenza) who confirmed:

   • Using "litellm_proxy/claude-3-5-sonnet-20241022-v2:0" now works correctly
   • The original BedrockException error no longer occurs with this configuration

4. Documentation was updated to:

   • Create a new guide for LiteLLM proxy usage
   • Explicitly recommend using the "litellm_proxy" prefix
   • Add references in the main LLMs documentation

The PR can be explained to reviewers as: "Fixed BedrockException error when using LiteLLM proxy by implementing proper provider prefix handling and adding comprehensive documentation for LiteLLM proxy configuration. This resolves issue #4912 where tool results were being incorrectly processed due to provider prefix mismatches."

Automatic fix generated by OpenHands 🙌

---

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:b08b067-nikolaik   --name openhands-app-b08b067   docker.all-hands.dev/all-hands-ai/openhands:b08b067

[neubig]

Just out of curiosity, why is this?

[xingyaoww]

This is a weird issue we found in #4912 -- not sure why 😓 but it seems to get rid of the bedrock bug

[enyst]

Does this mean that the users are supposed to run the litellm proxy? In my understanding, the 'openai/' prefix works regardless whether there's a litellm proxy on the way or not.

[enyst]

Ah I see. It might be that litellm proxy needs it now, and I'm not sure if it's possible for our setup to change it, or litellm proxy now really requires it.

On the other hand, the openai/ prefix should still be usable. I just tried it with other providers, and it works as usual (like openrouter).

[mamoodi]

This shows how to set it in development workflow. Would it be better to follow the other provider documentations and show how to do it in the UI so documentation is consistent?
If it's too much, I can revise it afterward it has merged. Been meaning to do a consistency check in docs.

[xingyaoww]

Yeah sounds good.. can you modify this PR directly though?

[mamoodi]

Yes sir. I'll try to take a look by end of day.

[mamoodi]

@xingyaoww as promised, I made it consistent with the rest of the providers. And added to sidebar.