Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bedrock token count callbacks #20

Merged
merged 31 commits into from
May 21, 2024

Commits on Apr 20, 2024

  1. Updating _prepare_input_and_invoke to use the on_llm_end callback

    on_llm_end, invoke will provide the usage and stop_reason information
    NAPTlME committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    7a22060 View commit details
    Browse the repository at this point in the history
  2. Consolidated GenerationChunk creation to _stream_response_to_generati…

    …on_chunk
    
    Also allowed additional response_body information to be passed into the generation_info (such as token counts and stop reasons)
    NAPTlME committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    929e3db View commit details
    Browse the repository at this point in the history
  3. Added an on_llm_end call to the streaming route

    The on_llm_end call will include all GenerationChunks as well as info on usage and stop_reason (in the llm_output)
    
    Added a provider stop code mapping
    NAPTlME committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    9b840d1 View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. Updating async streaming options to match streaming functions wrt mes…

    …sages api and outputs
    NAPTlME committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    5802e53 View commit details
    Browse the repository at this point in the history
  2. Updating _stream_response_to_generation_chunk to get the stop reason …

    …from the correct dict
    NAPTlME committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    140333b View commit details
    Browse the repository at this point in the history
  3. Ending on "message_stop" rather that "content_block_stop" for message…

    …s api
    
    Need the `message_delta` to return in order to get the stop reason and output token count
    
    https://docs.anthropic.com/claude/reference/messages-streaming
    NAPTlME committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    2f2c8ed View commit details
    Browse the repository at this point in the history
  4. Updating _prepare_intput_and_invoke to return a packaged llm_output i…

    …ncluding the stop_reason rather than just usage_info
    
    Removing the on_llm_end that was added and moving to _call to match with the pattern used for the ChatModel (as this was causing on_llm_end to be called twice)
    NAPTlME committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    dc9c9cd View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    a91b3c7 View commit details
    Browse the repository at this point in the history
  6. _combine_generation_info_for_llm_result is now using the generation_i…

    …nfo rather than the full chunks to make it compatible with some calls needed from ChatBedrock
    
    Updating the usage info to use prompt_tokens and completion_tokens to match those returned from the non streaming route
    NAPTlME committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    0f345c3 View commit details
    Browse the repository at this point in the history
  7. changing to output.get() rather than pop() to prevent the usage from …

    …being cleared from the llm_output
    
    This was returning llm_outputs with no usage information
    NAPTlME committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    25e9caf View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    8006cb5 View commit details
    Browse the repository at this point in the history
  9. Getting llm_output (containing usage and stop_reason from both st…

    …reaming and non-streaming generation and passing into the ChatResult
    NAPTlME committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    1eeb60f View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    afda911 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    02893cc View commit details
    Browse the repository at this point in the history
  12. Line Length for Ruff

    NAPTlME committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    794f9e5 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    b4c5f61 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. Ruff Formatting

    NAPTlME committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    6a870de View commit details
    Browse the repository at this point in the history
  2. Fixing issue with path and branch specification in lint_diff/format_diff

    These appear to be holdovers from the langchain repo's structure and default branch
    NAPTlME committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    932204a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4750b80 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. Configuration menu
    Copy the full SHA
    8a2465c View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. Configuration menu
    Copy the full SHA
    64b3578 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. Configuration menu
    Copy the full SHA
    1526f5a View commit details
    Browse the repository at this point in the history

Commits on May 17, 2024

  1. Configuration menu
    Copy the full SHA
    7a7e39f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    98d6d09 View commit details
    Browse the repository at this point in the history
  3. Fixing codespell errors

    NAPTlME committed May 17, 2024
    Configuration menu
    Copy the full SHA
    0f681cb View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    f1350c4 View commit details
    Browse the repository at this point in the history
  5. Updating usage info for input/output tokens to use lists to contain t…

    …hese integer values to prevent errors when merge_dicts is called from langchain_core.utils.
    NAPTlME committed May 17, 2024
    Configuration menu
    Copy the full SHA
    8908cd8 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    aa33982 View commit details
    Browse the repository at this point in the history
  7. Adding the usage_info transformation in which the integers are put in…

    … lists to the message_start message type.
    NAPTlME committed May 17, 2024
    Configuration menu
    Copy the full SHA
    f5e4431 View commit details
    Browse the repository at this point in the history
  8. Moving the usage_info nesting of token counts into lists into a funct…

    …ion to reduce the code duplication.
    NAPTlME committed May 17, 2024
    Configuration menu
    Copy the full SHA
    5eace06 View commit details
    Browse the repository at this point in the history
  9. Ran into an issue where the `test_chat_bedrock_streaming_generation_i…

    …nfo` test would occasionally fail (returned "Hello! How are you doing?" one time)
    NAPTlME committed May 17, 2024
    Configuration menu
    Copy the full SHA
    bfa0871 View commit details
    Browse the repository at this point in the history