Collect, process and visualise statistics from zeropingheroes/lancache with Elasticsearch, Logstash and Kibana
These screenshots are taken from a 100 person event, with the cache pre-loaded using zeropingheroes/lancache-autofill
- Ubuntu Server 18.04
lancache-elk cannot be run on the same host as lancache as both use port 80.
-
git clone https://github.com/zeropingheroes/lancache-elk.git && cd lancache-elk -
cp .env.example .env -
nano .env- add a password for the web interface -
./install.sh -
Log into the Kibana web interface and go to Management > Saved Objects
-
Import
lancache-elk/configs/kibana/export.jsonchoosing "logstash-*" in the dropdown under "New index pattern" -
Repeat step 6 a second time - not sure exactly why this is necessary...
- Overall - Global cumulative data about the cache
- General - Visualisations that apply to any upstream
- Steam - See which depots are cached, and which Steam users are downloading games
Because we use slice for these upstreams, Nginx reports a cache hit for almost all requests from clients, as the single client request has spawned one or more subrequests which fill the cache.
This also means that our statistics for how much data has been added to the cache is incorrect too. If you have a workaround for this behaviour, please submit a pull request.
Check your ELK server's log files:
tail /var/log/logstash/logstash-plain.log
tail /var/log/elasticsearch/elasticsearch.log
journalctl -u kibana.serviceCheck your lancache server's log files:
tail /var/log/filebeat/filebeat
journalctl -u filebeat.servicePlease submit pull requests for improving the configuration files here.
For example:
- Extracting additional fields from URIs
- Useful Kibana visualisations
Useful sites: