Drain logs from cloud.gov into your custom logging solution
For deployment
- Cloud Foundry CLI (tested with v8)
- cf-drains-cli plugin (tested with v2.0.0)
- jq (tested with 1.6)
For development, add
Use the drain plugin to configure the log drain for each app in a space.
cf drain <app-name> https://<username>:<password>@<drain-app-route>
Alternatively, you can auto-drain all apps in a given space by targeting that space, then running the ./create-space-drain.sh script.
cf target -s prod
./create-space-drain.sh
Note: at development space, it will use the logstack-shipper in development-ssb space:
cf target -s development
./create-space-drain.sh development-ssb
After a short delay, logs should begin to flow automatically.
Set your application name.
app_name=logstack
Copy vars.example.yml to vars.yml (or a space-specific version) and
customize for your application.
Create an S3 bucket instance.
cf create-service s3 basic ${app_name}-s3
Create a user provided service for secrets.
Push the applications.
cf push --vars-file vars.yml
Provide secrets for the logstack applications via a user-provided service.
cf cups ${app_name}-secrets -p DRAIN_USER,DRAIN_PASSWORD
| Name | Description | Where to find? |
|---|---|---|
| DRAIN_PASSWORD | Password for basic authentication on the Logstash proxy | randomly generated |
| DRAIN_USER | Username for basic authentication on the Logstash proxy | randomly generated |
The logstack application is made up of several smaller Cloud Foundry applications.
| Name | Description |
|---|---|
| logstack-shipper | Logstash process that aggregates and parses log data. |
| logstack-space-drain | Space drain monitors a CF space, and binds the log drain to applications. Created by the drains plugin. |
Note: The logstack-space-drain application consumes 64MB.
Run tests.
docker compose run --rm test
See CONTRIBUTING for additional information.
This project is in the worldwide public domain. As stated in CONTRIBUTING:
This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication.
All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.