[Feature] Middleware Framework and First Middleware Integration

[yhbcode000]

Middleware: What is it?

Middleware refers to software that provides common services and capabilities to applications outside of what's offered by the operating system. It acts as a bridge, enabling communication and data management for distributed applications. In this case, the middleware sits between the LLM (Large Language Model) layer and the user UI layer.

Why do we need middleware between the LLM layer and the user UI layer?

Middleware is essential here to ensure prompt anonymization and token management between user inputs and the LLM. It intercepts the prompt from the user, processes it to identify sensitive information, and replaces sensitive tokens with placeholder or alternate tokens. The middleware also handles the reverse mapping, allowing for seamless bi-directional conversion of anonymized content.

First Anonymization Middleware Layer

The input format for the middleware is simple:

{
  "prompt": "PROMPT TO AI"
}

cURL Example:

curl -X 'POST' \
  'address:8000/process_prompt' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "prompt": "string"
}'

Expected Output:

The middleware will return the original prompt with target tokens that need to be replaced, along with a list of possible replacement tokens.

Feature Requirement:

• The middleware should keep a dictionary to map sensitive keys to replacement tokens.
• This dictionary should allow for bi-directional conversion between the original sensitive content and its anonymized version.

Reference:

• Check the group OneDrive for more information about the implementation.