Replace global dashboard with activity dashboard

[consideRatio]

The activity.jsonnet dashboard is extracting four panels from the jupyterhub.jsonnet dashboard. The running servers panel and daily/weekly/monhtly active users panels.

By doing this, we get a dashboard with only those four panels, grouped into a row. This row is then made repeatable over the prometheus datasource variable. With a repeating row, we could end up with very many rows, making any panels below hard to get to. Due to that, it makes sense to use repeating rows in a dedicated dashboard.

The global dashboard that is being deleted made use of hardcoded state in the grafana instance it was being deployed to. This then also required regular re-deploys of the dashboard just to update the hardcoded entries.

Further, the global dashboard couldn't be pre-rendered and re-used across grafana instances - because it had grafance instance specific state within it.

With the new Activity dashboard, we can avoid all this complexity and possibly publish pre-rendered dashboards that any grafana instance can install directly.

• Fixes Stop generating global dashboard using state in specific grafana instance #111
• Fixes Global dashboard - Use JupyterHub's active users metric instead of counting pods #118
• Fixes Expose Global Users Dashboard as a timeseries #78

Visual comparison

New dashboard (redacted)

Rows with four panels are repeating per datasource here.

Old dashboard (redacted)

An individual panel is having hardcoded queries to all datasources as found when deploying the dashboard - which required the deploy.py script as it fetches info about all datasources.

[choldgraf]

I really like the look of the new dashboard. If it makes things less brittle and more maintainable as well then I think it's worth merging once folks are happy with the implementation.

I still think it would be really helpful to have aggregate graphs (eg by stacking graphs across clusters into a single plot), but perhaps that's not possible directly within grafana?

Two relevant issues for that

• Expose Global Users Dashboard as a timeseries #78
• Expose a panel that combines user activity from all prometheus datasources #79

[consideRatio]

I still think it would be really helpful to have aggregate graphs (eg by stacking graphs across clusters into a single plot), but perhaps that's not possible directly within grafana?

Not without hardcoding datasources =/ The "single plot" (aka. panel in grafana terminology) is the crux - grafana doesn't support functionality to let one panel reference a variable amount of datasources in its queries. One query per datasource, and a variable amounts of queries isn't possible.

[choldgraf]

Then it sounds like the solution would be to document how one could manually grab the data from this dashboard in order to do whatever you want with it. Does that sound right? And does that sound complicated?

[choldgraf]

I'm approving the PR to make it clear that I don't think my comments above should block this one, as I think it's definitely an improvement! I can't comment on the implementation however, so I'll let @consideRatio and/or others decide how to move forward.

[GeorgianaElena]

Thanks for raising this @consideRatio. As the person introducing the global dashboards experiment, creating it was motivated by @choldgraf's need to have all clusters available in a panel. So, if @choldgraf is ok with removing it and build similar usage graphs outside of Grafana then go for it 👍🏼 I don' think anyone else used this dashboard anyway, and this was part of the reason for keeping out separated then the general ones.

Also, extracting the running servers panel and daily/weekly/monhtly active users panels in an activity dashboard sounds more intuitively to find I think.

Also, implementation wise everything looked good to me.

[consideRatio]

Thank you for reviewing @choldgraf and @GeorgianaElena!

With OK from @choldgraf and @GeorgianaElena, I'll go for a merge here.

I'm marking #78 as fixed as well with this, but leave #79 open as an issue to represent wanting to have a single panel to present activity from all prometheus datasources (could be k8s clusters as it is for 2i2c) - something that is believed to not to be possible without hardcoding things currently in the dashboard definition (or generating it with state from specific grafana instances).

With my JupyterHub hat on, I think its a bit out of scope for this project to provide a dashboard that combines multiple prometheus datasources to present a super-panel if it means doing the workarounds required to do it as currently needed.