docker-stats-logger

Sample output

'summary'. Similar to 'docker stats' command output

datetime;container_name;CPU%;MEM usage;MEM %;NET IN;NET OUT;BLOCK IN;BLOCK OUT
2023-08-07T11:44:11;project_service_1;0.8878904347826088;72876032;0.8852767509865571;25250101;31174151;0;0

'full'. API's output flattened with keys turned into columns

read;preread;pids_stats_current;blkio_stats_io_service_bytes_recursive_0_major;blkio_stats_io_service_bytes_recursive_0_minor;blkio_stats_io_service_bytes_recursive_0_op;blkio_stats_io_service_bytes_recursive_0_value;blkio_stats_io_service_bytes_recursive_1_major;blkio_stats_io_service_bytes_recursive_1_minor;blkio_stats_io_service_bytes_recursive_1_op;blkio_stats_io_service_bytes_recursive_1_value;blkio_stats_io_service_bytes_recursive_2_major;blkio_stats_io_service_bytes_recursive_2_minor;blkio_stats_io_service_bytes_recursive_2_op;blkio_stats_io_service_bytes_recursive_2_value;blkio_stats_io_service_bytes_recursive_3_major;blkio_stats_io_service_bytes_recursive_3_minor;blkio_stats_io_service_bytes_recursive_3_op;blkio_stats_io_service_bytes_recursive_3_value;blkio_stats_io_service_bytes_recursive_4_major;blkio_stats_io_service_bytes_recursive_4_minor;blkio_stats_io_service_bytes_recursive_4_op;blkio_stats_io_service_bytes_recursive_4_value;blkio_stats_io_service_bytes_recursive_5_major;blkio_stats_io_service_bytes_recursive_5_minor;blkio_stats_io_service_bytes_recursive_5_op;blkio_stats_io_service_bytes_recursive_5_value;blkio_stats_io_service_bytes_recursive_6_major;blkio_stats_io_service_bytes_recursive_6_minor;blkio_stats_io_service_bytes_recursive_6_op;blkio_stats_io_service_bytes_recursive_6_value;blkio_stats_io_service_bytes_recursive_7_major;blkio_stats_io_service_bytes_recursive_7_minor;blkio_stats_io_service_bytes_recursive_7_op;blkio_stats_io_service_bytes_recursive_7_value;blkio_stats_io_service_bytes_recursive_8_major;blkio_stats_io_service_bytes_recursive_8_minor;blkio_stats_io_service_bytes_recursive_8_op;blkio_stats_io_service_bytes_recursive_8_value;blkio_stats_io_service_bytes_recursive_9_major;blkio_stats_io_service_bytes_recursive_9_minor;blkio_stats_io_service_bytes_recursive_9_op;blkio_stats_io_service_bytes_recursive_9_value;blkio_stats_io_service_bytes_recursive_10_major;blkio_stats_io_service_bytes_recursive_10_minor;blkio_stats_io_service_bytes_recursive_10_op;blkio_stats_io_service_bytes_recursive_10_value;blkio_stats_io_service_bytes_recursive_11_major;blkio_stats_io_service_bytes_recursive_11_minor;blkio_stats_io_service_bytes_recursive_11_op;blkio_stats_io_service_bytes_recursive_11_value;blkio_stats_io_service_bytes_recursive_12_major;blkio_stats_io_service_bytes_recursive_12_minor;blkio_stats_io_service_bytes_recursive_12_op;blkio_stats_io_service_bytes_recursive_12_value;blkio_stats_io_service_bytes_recursive_13_major;blkio_stats_io_service_bytes_recursive_13_minor;blkio_stats_io_service_bytes_recursive_13_op;blkio_stats_io_service_bytes_recursive_13_value;blkio_stats_io_service_bytes_recursive_14_major;blkio_stats_io_service_bytes_recursive_14_minor;blkio_stats_io_service_bytes_recursive_14_op;blkio_stats_io_service_bytes_recursive_14_value;blkio_stats_io_service_bytes_recursive_15_major;blkio_stats_io_service_bytes_recursive_15_minor;blkio_stats_io_service_bytes_recursive_15_op;blkio_stats_io_service_bytes_recursive_15_value;blkio_stats_io_service_bytes_recursive_16_major;blkio_stats_io_service_bytes_recursive_16_minor;blkio_stats_io_service_bytes_recursive_16_op;blkio_stats_io_service_bytes_recursive_16_value;blkio_stats_io_service_bytes_recursive_17_major;blkio_stats_io_service_bytes_recursive_17_minor;blkio_stats_io_service_bytes_recursive_17_op;blkio_stats_io_service_bytes_recursive_17_value;blkio_stats_io_service_bytes_recursive_18_major;blkio_stats_io_service_bytes_recursive_18_minor;blkio_stats_io_service_bytes_recursive_18_op;blkio_stats_io_service_bytes_recursive_18_value;blkio_stats_io_service_bytes_recursive_19_major;blkio_stats_io_service_bytes_recursive_19_minor;blkio_stats_io_service_bytes_recursive_19_op;blkio_stats_io_service_bytes_recursive_19_value;blkio_stats_io_service_bytes_recursive_20_major;blkio_stats_io_service_bytes_recursive_20_minor;blkio_stats_io_service_bytes_recursive_20_op;blkio_stats_io_service_bytes_recursive_20_value;blkio_stats_io_service_bytes_recursive_21_major;blkio_stats_io_service_bytes_recursive_21_minor;blkio_stats_io_service_bytes_recursive_21_op;blkio_stats_io_service_bytes_recursive_21_value;blkio_stats_io_service_bytes_recursive_22_major;blkio_stats_io_service_bytes_recursive_22_minor;blkio_stats_io_service_bytes_recursive_22_op;blkio_stats_io_service_bytes_recursive_22_value;blkio_stats_io_service_bytes_recursive_23_major;blkio_stats_io_service_bytes_recursive_23_minor;blkio_stats_io_service_bytes_recursive_23_op;blkio_stats_io_service_bytes_recursive_23_value;blkio_stats_io_serviced_recursive_0_major;blkio_stats_io_serviced_recursive_0_minor;blkio_stats_io_serviced_recursive_0_op;blkio_stats_io_serviced_recursive_0_value;blkio_stats_io_serviced_recursive_1_major;blkio_stats_io_serviced_recursive_1_minor;blkio_stats_io_serviced_recursive_1_op;blkio_stats_io_serviced_recursive_1_value;blkio_stats_io_serviced_recursive_2_major;blkio_stats_io_serviced_recursive_2_minor;blkio_stats_io_serviced_recursive_2_op;blkio_stats_io_serviced_recursive_2_value;blkio_stats_io_serviced_recursive_3_major;blkio_stats_io_serviced_recursive_3_minor;blkio_stats_io_serviced_recursive_3_op;blkio_stats_io_serviced_recursive_3_value;blkio_stats_io_serviced_recursive_4_major;blkio_stats_io_serviced_recursive_4_minor;blkio_stats_io_serviced_recursive_4_op;blkio_stats_io_serviced_recursive_4_value;blkio_stats_io_serviced_recursive_5_major;blkio_stats_io_serviced_recursive_5_minor;blkio_stats_io_serviced_recursive_5_op;blkio_stats_io_serviced_recursive_5_value;blkio_stats_io_serviced_recursive_6_major;blkio_stats_io_serviced_recursive_6_minor;blkio_stats_io_serviced_recursive_6_op;blkio_stats_io_serviced_recursive_6_value;blkio_stats_io_serviced_recursive_7_major;blkio_stats_io_serviced_recursive_7_minor;blkio_stats_io_serviced_recursive_7_op;blkio_stats_io_serviced_recursive_7_value;blkio_stats_io_serviced_recursive_8_major;blkio_stats_io_serviced_recursive_8_minor;blkio_stats_io_serviced_recursive_8_op;blkio_stats_io_serviced_recursive_8_value;blkio_stats_io_serviced_recursive_9_major;blkio_stats_io_serviced_recursive_9_minor;blkio_stats_io_serviced_recursive_9_op;blkio_stats_io_serviced_recursive_9_value;blkio_stats_io_serviced_recursive_10_major;blkio_stats_io_serviced_recursive_10_minor;blkio_stats_io_serviced_recursive_10_op;blkio_stats_io_serviced_recursive_10_value;blkio_stats_io_serviced_recursive_11_major;blkio_stats_io_serviced_recursive_11_minor;blkio_stats_io_serviced_recursive_11_op;blkio_stats_io_serviced_recursive_11_value;blkio_stats_io_serviced_recursive_12_major;blkio_stats_io_serviced_recursive_12_minor;blkio_stats_io_serviced_recursive_12_op;blkio_stats_io_serviced_recursive_12_value;blkio_stats_io_serviced_recursive_13_major;blkio_stats_io_serviced_recursive_13_minor;blkio_stats_io_serviced_recursive_13_op;blkio_stats_io_serviced_recursive_13_value;blkio_stats_io_serviced_recursive_14_major;blkio_stats_io_serviced_recursive_14_minor;blkio_stats_io_serviced_recursive_14_op;blkio_stats_io_serviced_recursive_14_value;blkio_stats_io_serviced_recursive_15_major;blkio_stats_io_serviced_recursive_15_minor;blkio_stats_io_serviced_recursive_15_op;blkio_stats_io_serviced_recursive_15_value;blkio_stats_io_serviced_recursive_16_major;blkio_stats_io_serviced_recursive_16_minor;blkio_stats_io_serviced_recursive_16_op;blkio_stats_io_serviced_recursive_16_value;blkio_stats_io_serviced_recursive_17_major;blkio_stats_io_serviced_recursive_17_minor;blkio_stats_io_serviced_recursive_17_op;blkio_stats_io_serviced_recursive_17_value;blkio_stats_io_serviced_recursive_18_major;blkio_stats_io_serviced_recursive_18_minor;blkio_stats_io_serviced_recursive_18_op;blkio_stats_io_serviced_recursive_18_value;blkio_stats_io_serviced_recursive_19_major;blkio_stats_io_serviced_recursive_19_minor;blkio_stats_io_serviced_recursive_19_op;blkio_stats_io_serviced_recursive_19_value;blkio_stats_io_serviced_recursive_20_major;blkio_stats_io_serviced_recursive_20_minor;blkio_stats_io_serviced_recursive_20_op;blkio_stats_io_serviced_recursive_20_value;blkio_stats_io_serviced_recursive_21_major;blkio_stats_io_serviced_recursive_21_minor;blkio_stats_io_serviced_recursive_21_op;blkio_stats_io_serviced_recursive_21_value;blkio_stats_io_serviced_recursive_22_major;blkio_stats_io_serviced_recursive_22_minor;blkio_stats_io_serviced_recursive_22_op;blkio_stats_io_serviced_recursive_22_value;blkio_stats_io_serviced_recursive_23_major;blkio_stats_io_serviced_recursive_23_minor;blkio_stats_io_serviced_recursive_23_op;blkio_stats_io_serviced_recursive_23_value;num_procs;cpu_stats_cpu_usage_total_usage;cpu_stats_cpu_usage_percpu_usage_0;cpu_stats_cpu_usage_percpu_usage_1;cpu_stats_cpu_usage_percpu_usage_2;cpu_stats_cpu_usage_percpu_usage_3;cpu_stats_cpu_usage_usage_in_kernelmode;cpu_stats_cpu_usage_usage_in_usermode;cpu_stats_system_cpu_usage;cpu_stats_online_cpus;cpu_stats_throttling_data_periods;cpu_stats_throttling_data_throttled_periods;cpu_stats_throttling_data_throttled_time;precpu_stats_cpu_usage_total_usage;precpu_stats_cpu_usage_usage_in_kernelmode;precpu_stats_cpu_usage_usage_in_usermode;precpu_stats_throttling_data_periods;precpu_stats_throttling_data_throttled_periods;precpu_stats_throttling_data_throttled_time;memory_stats_usage;memory_stats_max_usage;memory_stats_stats_active_anon;memory_stats_stats_active_file;memory_stats_stats_cache;memory_stats_stats_dirty;memory_stats_stats_hierarchical_memory_limit;memory_stats_stats_hierarchical_memsw_limit;memory_stats_stats_inactive_anon;memory_stats_stats_inactive_file;memory_stats_stats_mapped_file;memory_stats_stats_pgfault;memory_stats_stats_pgmajfault;memory_stats_stats_pgpgin;memory_stats_stats_pgpgout;memory_stats_stats_rss;memory_stats_stats_rss_huge;memory_stats_stats_total_active_anon;memory_stats_stats_total_active_file;memory_stats_stats_total_cache;memory_stats_stats_total_dirty;memory_stats_stats_total_inactive_anon;memory_stats_stats_total_inactive_file;memory_stats_stats_total_mapped_file;memory_stats_stats_total_pgfault;memory_stats_stats_total_pgmajfault;memory_stats_stats_total_pgpgin;memory_stats_stats_total_pgpgout;memory_stats_stats_total_rss;memory_stats_stats_total_rss_huge;memory_stats_stats_total_unevictable;memory_stats_stats_total_writeback;memory_stats_stats_unevictable;memory_stats_stats_writeback;memory_stats_limit;name;id;networks_eth0_rx_bytes;networks_eth0_rx_packets;networks_eth0_rx_errors;networks_eth0_rx_dropped;networks_eth0_tx_bytes;networks_eth0_tx_packets;networks_eth0_tx_errors;networks_eth0_tx_dropped
2023-08-07T11:39:38.703071767Z;0001-01-01T00:00:00Z;24;253;2;Read;0;253;2;Write;0;253;2;Sync;0;253;2;Async;0;253;2;Discard;0;253;2;Total;0;8;0;Read;94277632;8;0;Write;0;8;0;Sync;94277632;8;0;Async;0;8;0;Discard;0;8;0;Total;94277632;253;0;Read;94277632;253;0;Write;1187840;253;0;Sync;95465472;253;0;Async;0;253;0;Discard;0;253;0;Total;95465472;253;1;Read;94277632;253;1;Write;1187840;253;1;Sync;95465472;253;1;Async;0;253;1;Discard;0;253;1;Total;95465472;253;2;Read;0;253;2;Write;0;253;2;Sync;0;253;2;Async;0;253;2;Discard;0;253;2;Total;0;8;0;Read;6060;8;0;Write;60;8;0;Sync;6120;8;0;Async;0;8;0;Discard;0;8;0;Total;6120;253;0;Read;6060;253;0;Write;206;253;0;Sync;6266;253;0;Async;0;253;0;Discard;0;253;0;Total;6266;253;1;Read;6060;253;1;Write;206;253;1;Sync;6266;253;1;Async;0;253;1;Discard;0;253;1;Total;6266;0;225919276880;56925665421;56006917965;56786481302;56200212192;48500000000;168740000000;90074860000000;4;0;0;0;0;0;0;0;0;0;73445376;143683584;11354112;6426624;10498048;8192;9223372036854771712;9223372036854771712;39759872;4071424;8241152;3543946;674;3539124;3524082;50941952;0;11354112;6426624;10498048;8192;39759872;4071424;8241152;3543946;674;3539124;3524082;50941952;0;0;0;0;0;8232005632;/project_service_1;6a4c55fd1cd81bab44b7562cbbedc770831059bdcfae8f43115c04ed49c47a50;24946168;217747;0;0;30763655;426817;0;0

'raw'. API's response unaltered

{'read': '2023-08-07T11:40:26.428871754Z', 'preread': '0001-01-01T00:00:00Z', 'pids_stats': {'current': 24}, 'blkio_stats': {'io_service_bytes_recursive': [{'major': 253, 'minor': 2, 'op': 'Read', 'value': 0}, {'major': 253, 'minor': 2, 'op': 'Write', 'value': 0}, {'major': 253, 'minor': 2, 'op': 'Sync', 'value': 0}, {'major': 253, 'minor': 2, 'op': 'Async', 'value': 0}, {'major': 253, 'minor': 2, 'op': 'Discard', 'value': 0}, {'major': 253, 'minor': 2, 'op': 'Total', 'value': 0}, {'major': 8, 'minor': 0, 'op': 'Read', 'value': 94277632}, {'major': 8, 'minor': 0, 'op': 'Write', 'value': 0}, {'major': 8, 'minor': 0, 'op': 'Sync', 'value': 94277632}, {'major': 8, 'minor': 0, 'op': 'Async', 'value': 0}, {'major': 8, 'minor': 0, 'op': 'Discard', 'value': 0}, {'major': 8, 'minor': 0, 'op': 'Total', 'value': 94277632}, {'major': 253, 'minor': 0, 'op': 'Read', 'value': 94277632}, {'major': 253, 'minor': 0, 'op': 'Write', 'value': 1187840}, {'major': 253, 'minor': 0, 'op': 'Sync', 'value': 95465472}, {'major': 253, 'minor': 0, 'op': 'Async', 'value': 0}, {'major': 253, 'minor': 0, 'op': 'Discard', 'value': 0}, {'major': 253, 'minor': 0, 'op': 'Total', 'value': 95465472}, {'major': 253, 'minor': 1, 'op': 'Read', 'value': 94277632}, {'major': 253, 'minor': 1, 'op': 'Write', 'value': 1187840}, {'major': 253, 'minor': 1, 'op': 'Sync', 'value': 95465472}, {'major': 253, 'minor': 1, 'op': 'Async', 'value': 0}, {'major': 253, 'minor': 1, 'op': 'Discard', 'value': 0}, {'major': 253, 'minor': 1, 'op': 'Total', 'value': 95465472}], 'io_serviced_recursive': [{'major': 253, 'minor': 2, 'op': 'Read', 'value': 0}, {'major': 253, 'minor': 2, 'op': 'Write', 'value': 0}, {'major': 253, 'minor': 2, 'op': 'Sync', 'value': 0}, {'major': 253, 'minor': 2, 'op': 'Async', 'value': 0}, {'major': 253, 'minor': 2, 'op': 'Discard', 'value': 0}, {'major': 253, 'minor': 2, 'op': 'Total', 'value': 0}, {'major': 8, 'minor': 0, 'op': 'Read', 'value': 6060}, {'major': 8, 'minor': 0, 'op': 'Write', 'value': 60}, {'major': 8, 'minor': 0, 'op': 'Sync', 'value': 6120}, {'major': 8, 'minor': 0, 'op': 'Async', 'value': 0}, {'major': 8, 'minor': 0, 'op': 'Discard', 'value': 0}, {'major': 8, 'minor': 0, 'op': 'Total', 'value': 6120}, {'major': 253, 'minor': 0, 'op': 'Read', 'value': 6060}, {'major': 253, 'minor': 0, 'op': 'Write', 'value': 206}, {'major': 253, 'minor': 0, 'op': 'Sync', 'value': 6266}, {'major': 253, 'minor': 0, 'op': 'Async', 'value': 0}, {'major': 253, 'minor': 0, 'op': 'Discard', 'value': 0}, {'major': 253, 'minor': 0, 'op': 'Total', 'value': 6266}, {'major': 253, 'minor': 1, 'op': 'Read', 'value': 6060}, {'major': 253, 'minor': 1, 'op': 'Write', 'value': 206}, {'major': 253, 'minor': 1, 'op': 'Sync', 'value': 6266}, {'major': 253, 'minor': 1, 'op': 'Async', 'value': 0}, {'major': 253, 'minor': 1, 'op': 'Discard', 'value': 0}, {'major': 253, 'minor': 1, 'op': 'Total', 'value': 6266}], 'io_queue_recursive': [], 'io_service_time_recursive': [], 'io_wait_time_recursive': [], 'io_merged_recursive': [], 'io_time_recursive': [], 'sectors_recursive': []}, 'num_procs': 0, 'storage_stats': {}, 'cpu_stats': {'cpu_usage': {'total_usage': 226453550759, 'percpu_usage': [57040873235, 56114414393, 56973845286, 56324417845], 'usage_in_kernelmode': 48610000000, 'usage_in_usermode': 169140000000}, 'system_cpu_usage': 90259860000000, 'online_cpus': 4, 'throttling_data': {'periods': 0, 'throttled_periods': 0, 'throttled_time': 0}}, 'precpu_stats': {'cpu_usage': {'total_usage': 0, 'usage_in_kernelmode': 0, 'usage_in_usermode': 0}, 'throttling_data': {'periods': 0, 'throttled_periods': 0, 'throttled_time': 0}}, 'memory_stats': {'usage': 73871360, 'max_usage': 143683584, 'stats': {'active_anon': 11354112, 'active_file': 6832128, 'cache': 10313728, 'dirty': 8192, 'hierarchical_memory_limit': 9223372036854771712, 'hierarchical_memsw_limit': 9223372036854771712, 'inactive_anon': 40349696, 'inactive_file': 3481600, 'mapped_file': 8192000, 'pgfault': 3551851, 'pgmajfault': 674, 'pgpgin': 3547070, 'pgpgout': 3531924, 'rss': 51552256, 'rss_huge': 0, 'total_active_anon': 11354112, 'total_active_file': 6832128, 'total_cache': 10313728, 'total_dirty': 8192, 'total_inactive_anon': 40349696, 'total_inactive_file': 3481600, 'total_mapped_file': 8192000, 'total_pgfault': 3551851, 'total_pgmajfault': 674, 'total_pgpgin': 3547070, 'total_pgpgout': 3531924, 'total_rss': 51552256, 'total_rss_huge': 0, 'total_unevictable': 0, 'total_writeback': 0, 'unevictable': 0, 'writeback': 0}, 'limit': 8232005632}, 'name': '/project_service_1', 'id': '6a4c55fd1cd81bab44b7562cbbedc770831059bdcfae8f43115c04ed49c47a50', 'networks': {'eth0': {'rx_bytes': 24998968, 'rx_packets': 218250, 'rx_errors': 0, 'rx_dropped': 0, 'tx_bytes': 30834965, 'tx_packets': 427806, 'tx_errors': 0, 'tx_dropped': 0}}}

Docker image env

Variable	Usage	Default
LOGGING_DIRECTORY	Directory where log files will be stored.	/usr/src/app/stats_logs/
LOGGING_EXCLUDE	Names of containers to ignore.
LOGGING_INCLUDE	Names of containers to monitor. Empty to log all of them. DOCKER_PROJECT and LOGGING_EXCLUDE have priority over this option.
LOGGING_INTERVAL	Interval between log entries in seconds. Due to the Docker API's response time it's impossible to get an interval time below 1 second.	0
LOGGING_MODE	Logging mode to use (see above for reference)	full
LOGGING_ONE_SHOT	Improve response time, decreasing the interval between entries. Ignored when LOGGING_MODE = 'summary'. See Docker API's documentation for more details: https://docs.docker.com/engine/api/v1.43/#tag/Container/operation/ContainerStats	false
DOCKER_PROJECT	Project name. Only containers whose name begins with the provided project name will be monitored. Otherwise, all containers will be monitored.

TODO

• Support/Test on Windows
• Make docker image
• Configure Github Actions
• Handle new containers and container restarts