Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

langchain_aws/llms/bedrock.py "stop_sequence" hardcoded for msg_type == "message_delta" - breaks with bedrock guardrails with langchain agents #214

Open
shayanrayamzn opened this issue Sep 24, 2024 · 1 comment

Comments

@shayanrayamzn
Copy link

Hello,

In langchain_aws/llms/bedrock.py the "stop_sequence" hardcoded for msg_type == "message_delta" -
Shouldn't this be captured using the LLM provider type with self._get_provider()

This breaks integration with bedrock guardrails and langchain agents when guardrails intervene




 elif msg_type == "message_delta":
            return AIMessageChunk(
                content="",
                response_metadata={
                    "stop_reason": stream_response["delta"]["stop_reason"],
                    **"stop_sequence": stream_response["delta"]["stop_sequence"],**
                },
            )

@langcarl langcarl bot added the investigate label Sep 24, 2024
@shayanrayamzn
Copy link
Author

shayanrayamzn commented Sep 24, 2024

If there are any workarounds for this fix please let me know. adding a provider_stop_sequence_key_name_map works for other issues but not in this case, when guardrails intervene and enter the msg_type == "message_delta" block

llm.provider_stop_sequence_key_name_map = {'anthropic': 'stop_sequences', 
                                                            'amazon': 'stopSequences',
                                                            'ai21': 'stop_sequences',
                                                            'cohere': 'stop_sequences',
                                                            'mistral': 'stop'}

@3coins 3coins added the bedrock label Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants