Environment Variables Not Accessible in Docker-based Tool Execution

[anu43]

Problem Description:

When using AutoGen with Docker-based code execution, there seems to be an issue with accessing environment variables within the executed functions. Specifically, when trying to use environment variables to store sensitive information like API keys, the variables are not accessible within the Docker container where the code is executed.

For example, consider the following scenario:

1. A weather API function is defined that uses environment variables for the API key and base URL:

def get_weather(location):
    api_key = os.getenv("OPENWEATHERMAP_API_KEY")
    base_url = os.getenv("OPENWEATHERMAP_BASE_URL")
    # Rest of the function...

2. The function is registered for use with AutoGen agents:

register_function(
    get_weather,
    caller=assistant,
    executor=user_proxy,
    name="get_weather",
    description="Get weather information for a given location",
)

3. The user_proxy agent is configured to use Docker for code execution:

user_proxy = ConversableAgent(
    name="User",
    code_execution_config={
        "executor": DockerCommandLineCodeExecutor(work_dir=work_dir),
        "work_dir": work_dir,
    },
    # Other configurations...
)

4. When the conversation is initiated and the weather function is called, it fails to access the environment variables within the Docker container.

However, if the API key and base URL are hardcoded directly in the function, it works correctly:

def get_weather(location):
    api_key = "actual_api_key_here"
    base_url = "https://api.openweathermap.org/data/2.5/weather"
    # Rest of the function...

This behavior suggests that environment variables set on the host machine are not being properly passed to or accessed within the Docker container during tool execution.

Questions:

1. Is this the expected behavior, or is it a bug?
2. If it's expected, what's the recommended way to securely pass sensitive information (like API keys) to functions executed in Docker containers with AutoGen?
3. Are there any configuration options or best practices for ensuring environment variables are accessible in Docker-based tool execution with AutoGen?

Any insights or guidance on this issue would be greatly appreciated, as it impacts the security and portability of AutoGen applications that rely on environment variables for configuration.

[anu43]

Would you happen to have any suggestions here?

[ekzhu]

@anu43 the docker code executor is used to run LLM generated code and is not used to execute the tools, which you have defined.

tool execution happens locally in your application's environment.

They are two separate code paths. So you will set the environment variables in your application's environment for your tools.

[ekzhu]

For docker based code execution, the best practice is to not set any environment variables in the execution environment due to arbitrary code can be executed.

If the code executor needs access to external API, it is best to use proxy to protect the secrets, or use process level managed identity, without any secret.

[anu43]

@ekzhu That makes sense, thanks! I am also wondering if there could be a way to include an option for passing an environment file path to the DockerCommandLineCodeExecutor. This file would then be copied into the container during the build process to execute the code.

[ekzhu]

The risk of exposing the content of the file still exists as long as arbitrary code execution is possible.