Cos proxy is sending to prometheus targets, from only one prometheus-target relation

[err404r]

Bug Description

We have several telegraf apps related to prometheus-target interface of cos-proxy.
On prometheus we could see data only from first telegraf unit. As well as within relation data
On cos proxy unit I could see data from relations to all telegraf units

To Reproduce

Relate several telegraf units to one cos-proxy

Environment

Charm revision 74

Relevant log output

Unfortunately no logs to add

Additional context

No response

[sed-i]

Does the deployment look something like this?

graph LR

subgraph LXD
telegraf ---|http| cos-proxy
telegraf ---|prometheus-rules| cos-proxy
end

subgraph COS
prometheus
end

cos-proxy ---|prometheus_scrape| prometheus

Loading

[err404r]

yes, but we have several telegraf applications related to one cos-proxy

[lucabello]

We should try to reproduce this issue with a few telegraf deployments. It might be just a timing issue, with cos-proxy taking a long time to settle with too many telegraf deployments.