-
Notifications
You must be signed in to change notification settings - Fork 214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Observability for RayServe and vLLM GPU #642
Conversation
feat: Rayserve and vLLM o11y
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great @shivam-dubey-1 ! Left few minor comments
enable_aws_cloudwatch_metrics = true | ||
aws_cloudwatch_metrics = { | ||
values = [templatefile("${path.module}/helm-values/aws-cloudwatch-metrics-values.yaml", {})] | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should switch to EKS managed add-on for CloudWatch and remove this. Please see the BioNemo PR as an example https://github.com/awslabs/data-on-eks/pull/641/files#diff-0222ef610acbd44e74c858a5505a877ad82439b934b0667e3f81dc91d3a247fdR26
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can do this in the second PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Verify pre-commit Min TF check please |
What does this PR do?
This PR provides observability with Prom and Grafana for rayserve-vllm-gpu pattern
🛑 Please open an issue first to discuss any significant work and flesh out details/direction - we would hate for your time to be wasted.
Consult the CONTRIBUTING guide for submitting pull-requests.
Motivation
#586
#645
More
website/docs
orwebsite/blog
section for this featurepre-commit run -a
with this PR. Link for installing pre-commit locallyFor Moderators
Additional Notes