Enabling RAG makes it relatively easy to reach 400KB limit for a dynamodb item on sessions table

[gbone-restore]

Amazon DynamoDB limits the size of each item in a table to 400 KB. It's feasible that this limit could be reached on the sessions table because of all the extra metadata stored in the item. Especially when running with RAG configured. The session table stores chat history in a single item attribute.

To work around this limitation, we can store chat history as individual items in the sessions table. I can submit a PR for this change if there is an interest in this issue.

[charles-marion]

Hi @gbone-restore ,

I agree it is an issue but changing the table structure might be a significant change.

A shorter/smaller change could be to reduce the metadata stored and limit the number of message in the sessions.

[gbone-restore]

Here's the data model that I used to test out this change.

The partition key is now users and the sort key is sessions. And there are two types of items to make selecting session history and dealing with chat history a little easier. I can submit a PR with a working example. The bulk of the changes are in lib/shared/layers/python-sdk/python/genai_core/langchain/chat_message_history.py and lib/shared/layers/python-sdk/python/genai_core/sessions.py.

[charles-marion]

The partition key is now users and the sort key is sessions.

I think it's a good solution. My concern is it would invalidate existing sessions

I can submit a PR with a working example

If you already have a prototype available it would be welcome.

Note I intend to discuss this topic with the maintainers.