Docker logs: Distinction between stdout and stderr in correct order

[alfechner]

I need to stream the logs from my docker container for logging.

Currently I do this:

stream = container.logs(stdout=True, stderr=True, stream=True)

for line in stream:
    logger.info(line.decode("utf-8"))

Now I need to differentiate between stdout and stderr. The stdout is supposed to be logged to logger.info, the stderr to logger.error.

My approach is to setup two streams, one for stdout and one for stderr and read them in parallel with different threads.

def process_stream(stream, callback):
    for line in stream:
        callback(line.decode("utf-8").strip())

stdout_stream = container.logs(stdout=True, stderr=False, stream=True, timestamps=True)
stderr_stream = container.logs(stdout=False, stderr=True, stream=True, timestamps=True)

stdout_thread = Thread(
    target=process_stream,
    args=(
        stdout_stream,
        logger.info,
    ),
)
stderr_thread = Thread(
    target=process_stream,
    args=(
        stderr_stream,
        logger.error,
    ),
)

stdout_thread.start()
stderr_thread.start()

stdout_thread.join()
stderr_thread.join()

However, messages from stdout and stderr are not always logged in correct order.

I also looked into ways to access the log files from the Docker JSON File logging driver I'm using directly but couldn't find a way.

Is there a way stream the messages in correct order plus the information if the message came from stdout or stderr?

[Khushiyant]

A better approach would be to use a single stream with the demux parameter set to True, which allows you to differentiate between stdout and stderr logs.

[alfechner]

Hi @Khushiyant,

thanks so much for your reply and the hint to use demux. This parameter is available for the exec_run method but not for the logs method I'm using.

The main difference is, that I need to intercept the stream. exec_run returns stdout and stderr as bytes after the execution is fully done. I need to see the output right when it happens.

[Khushiyant]

Ok, I get it. Let's what I can do. But, It might be delayed since it is not on next release priority