Skip to content
This repository has been archived by the owner on Aug 30, 2024. It is now read-only.

cloudwatch 3.3 cloud_design

Jump to bottom
chris grzegorczyk edited this page Feb 1, 2013 · 6 revisions

Table of Contents

Overview

Implement an AWS compatible Cloud Watch solution.

Out of Scope

VPC, SNS, DynamoDB, Billing, ElastiCache, ElasticMapReduce, RDS, SQS, StorageGateway

Feature Dependencies

Ground true data collection per each system-defined metric.

Related Features

AutoScaling, ELB

Design

Entity Model

Database: eucalyptus_cloudwatch

Tables: metric_data (used for PutMetricData/GetMetricStatistics) list_metrics (metrics for ListMetrics) list_metrics_dimensions

CRUD design

metric_data

Create : An aggregation of raw data rows will be persisted with the following column definitions :

Per row entry (hash code of each row for lookup == Dimensions (key:value)) - user, account, uuid, MetricName, MetricType(Predefined or User Custom), MetricActions?, (string), NameSpace, timestamp, SUM, sample points, max, min

(Note, timestamp format : http://en.wikipedia.org/wiki/ISO_8601 : Timestamps to be truncated to the previous minute)

Read : The key for search will be based on the a hash code that represents a possible combination of dimension (key:value), metric name, and namespace

Update : Not required

Delete - removes rows in which have a time stamp that is equals (current system time - (current time + two weeks))

list_metrics

Create : The metric info (without metric value), will be persisted persisted with the following column definitions :

Per row entry (hash code of each row for uniqueness, including user, account, MetricName, namespace, and dimensions (key:value)) - user, account, uuid, MetricName, NameSpace, timestamp (create and update) Create will also cascade into list_metrics_dimensions

Read: Searches can be based on metric name, namespace, or a dimension list (dimension subset match is allowed). Result will include the entire dimension list entered with the metric

Update: Only the update timestamp of the list_metrics row will be updated

Delete - remove rows that have an update time stamp that is equals (current system time - (current time + two weeks)). list_metric_dimensions will cascade delete.

Uniqueness_Constraints: Only the combination of account_id, user_id, metric_name, namespace, and dimension key/values (in order) need to be unique. This is the purpose of the hash.

list_metrics_dimensions

Create: A cascade insert from list_metrics will be persisted with the following column definitions: dimension_name, dimension_value, metric_id

Read: Only supported through join with list_metrics

Update: Not supported

Delete: only supported through cascade delete with list_metrics

Service Impact

CLC -> Consume general jvm resource the cloud controller

DB -> additional jdbc connections

Alarms -> additional duty cycle in which will spawn more threads

Must be HA compliant

Put Metric Workflow

put-metric-data-workflow.png

Put Metric Workflow Diagram Definitions

  • PutMetricData -> user created request
  • Cloud Watch Service -> Eucalyptus implementation of the Cloud Watch Service
  • Raw Data Queue -> FIFO queue
  • Aggregation -> Function to process an aggregation of one minute window of raw data to be inserted into the database
  • Cleaner -> House keeping process to delete metric data from the database with a 2 week window
  • Data Points Rows -> Processed information

List Metrics Workflow

list-metrics-workflow.png

List Metrics Workflow Diagram Definitions

  • ListMetrics -> user created request
  • Cloud Watch Service -> Eucalyptus implementation of the Cloud Watch Service
  • Raw Data Queue -> FIFO queue
  • Aggregation -> Function to process an aggregation of one minute window of raw data to be inserted into the database (might use the same workflow as PutMetricData)
  • Cleaner -> House keeping process to delete metric data from the database with a 2 week window
  • list_metrics and list_metrics_data -> Processed information (Any exact metric match in the database will have its timestamp updated)

Database Purge

Global period data deletion after 2 weeks.

Discovery

Filters should be discovered, whether individually, per type or per API.

Upgrade

No upgrade impact noted.

Packaging

No specific packaging requirements.

Documentation

  • Need feedback from the docs team

Security

No specific security concerns are noted for this design.

Testing

Testing should cover SOAP and Query APIs

References

AWS Cloud Watch : http://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/Welcome.html JIRA https://eucalyptus.atlassian.net/browse/EUCA-4307

Clone this wiki locally