The datadog-agent service is not being registered when installing on Amazon Linux

[jtstrohl]

After seemingly successful install of the datadog cookbook version 4.3.0 using recipes dd-agent and dd-handler, I can see that the Datadog agent is running and showing up in the Datadog web console. However, according to the Datadog documentation here, we should be able to run the following Linux command to get version and other information about the installed DD agent:

sudo service datadog-agent status

However, we get the following output when running this command:

datadog-agent: unrecognized service

Also, the service configuration no longer shows up in /etc/init.d/datadog-agent like it used to when installed with previous versions of this cookbook.

By all indications in the chef-client output, the service is being set up and started, configured, and restarted. What gives?

[2020-06-08T21:36:58+00:00] INFO: Processing directory[/etc/datadog-agent] action create (datadog::dd-agent line 60)
[2020-06-08T21:36:58+00:00] INFO: Processing service[datadog-agent] action enable (datadog::dd-agent line 144)
[2020-06-08T21:36:58+00:00] INFO: Processing service[datadog-agent] action start (datadog::dd-agent line 144)
[2020-06-08T21:36:58+00:00] INFO: service[datadog-agent] started
.
.
.
[2020-06-08T21:37:02+00:00] INFO: template[/etc/datadog-agent/datadog.yaml] sending restart action to service[datadog-agent] (delayed)
[2020-06-08T21:37:02+00:00] INFO: Processing service[datadog-agent] action restart (datadog::dd-agent line 144)
[2020-06-08T21:37:07+00:00] INFO: service[datadog-agent] restarted

[albertvaka]

Hi @jtstrohl!

Amazon Linux (at least, AL2) uses systemd, so the init config doesn't need to be in /etc/init.d/datadog-agent for it to work.

Can you run sudo systemctl daemon-reload then sudo systemctl status datadog-agent and see if this works?

Also, what version of the Agent are you installing (if not latest)? You can see that with datadog-agent version

[jtstrohl]

Hi @albertvaka -

Thanks for your response....We are using Amazon Linux 1 which does NOT use systemd, so unfortunately, I cannot run the systemctl command because the command doesn't exist on this version.

The funny thing is, we build off of the same AWS AMI every time. When we were using datadog cookbook version 2.16.0, the datadog-agent service installs successfully, and the /etc/init.d/datadog-agent file is there, and the service responds to commands like sudo service datadog-agent status. However, when we use the latest datadog cookbook version 4.3.0, the recipe completes successfully, but the /etc/init.d/datadog-agent file is NOT there, and the service does NOT respond to commands like sudo service datadog-agent status.

As you can see in the below snippet, there is indeed a datadog agent process running, but it seems to launched by the recipe using a bash script -- We want to understand why it doesn't respond to the "service" interface, because that's how the official Datadog Linux documentation gives instructions to inspect/manage/maintain the service.

# ps -ef |grep dd-agent
root      5248     1  0 Jun08 ?        00:00:00 su -s /bin/sh -c DD_LOG_TO_CONSOLE=false exec "$0" "$@" dd-agent -- /opt/datadog-agent/bin/agent/agent run -p /opt/datadog-agent/run/agent.pid
root      5260     1  0 Jun08 ?        00:00:00 su -s /bin/sh -c DD_LOG_TO_CONSOLE=false exec "$0" "$@" dd-agent -- /opt/datadog-agent/embedded/bin/process-agent --config=/etc/datadog-agent/datadog.yaml --sysprobe-config=/etc/datadog-agent/system-probe.yaml --pid=/opt/datadog-agent/run/process-agent.pid
dd-agent  5261  5248  0 Jun08 ?        00:07:46 /opt/datadog-agent/bin/agent/agent run -p /opt/datadog-agent/run/agent.pid
root      5267     1  0 Jun08 ?        00:00:00 su -s /bin/sh -c DD_LOG_TO_CONSOLE=false exec "$0" "$@" dd-agent -- /opt/datadog-agent/embedded/bin/trace-agent --config /etc/datadog-agent/datadog.yaml --pid /opt/datadog-agent/run/trace-agent.pid
dd-agent  5269  5260  0 Jun08 ?        00:00:30 /opt/datadog-agent/embedded/bin/process-agent --config=/etc/datadog-agent/datadog.yaml --sysprobe-config=/etc/datadog-agent/system-probe.yaml --pid=/opt/datadog-agent/run/process-agent.pid
dd-agent  5275  5267  0 Jun08 ?        00:01:12 /opt/datadog-agent/embedded/bin/trace-agent --config /etc/datadog-agent/datadog.yaml --pid /opt/datadog-agent/run/trace-agent.pid

[albertvaka]

Hi again @jtstrohl,

I think I know what happens here. Cookbook 2.x installed Agent 5 by default, while the current cookbook installs Agent 7. While the Agent 5 RPM package included SysV init files, it's not the case for the Agent 7 RPM package.

I don't think there is enough demand for SysV init files to justify adding them at this point, but I will pass it as a feature request for the Agent team. And about the docs: we should fix them since as you point out they don't seem to be updated.

Albert

[albertvaka]

Reading a bit about AL1, it seems to use upstart and not SysV. In that case, the following command should work: sudo initctl status datadog-agent

[jtstrohl]

Thanks @albertvaka .... That explanation makes a lot of sense actually -- So the difference is in the agent RPM package, not with the cookbook/recipe or their dependencies.

AL1 does appear to support upstart, although the initctl command does not exist on my instance. Alternatively, I do have /sbin/init and /sbin/status.

I also discovered that I can run the following commands respectively to start, stop, and status the agent process, even though it's not as ideal as being able to use the SysV commands:

• sudo datadog-agent run
• sudo datadog-agent stop
• sudo datadog-agent status

And even these commands and their syntax differ a bit from what is included in the Datadog documentation under the "Other Commands" section.

So aside from the fact that I think the Documentation probably needs updated, I think the case can be closed as there is technically no "issue" with the cookbook/recipe. What are your thoughts?

I appreciate your attention and help on this.

--Jonathan

[albertvaka]

I'm happy that works for you :) Note that datadog-agent run will make it run on the foreground, though (as compared to SysV commands, that would spawn a background process). We will update the docs to reflect platform-specific commands.