Enhance kinesis Consumer support #200
Conversation
zzhlogin
changed the title
aws-opentelemetry-distro/src/amazon/opentelemetry/distro/_aws_attribute_keys.py
Outdated
Show resolved
Hide resolved
...pentelemetry-distro/tests/amazon/opentelemetry/distro/test_aws_metric_attribute_generator.py
Outdated
Show resolved
Hide resolved
aws-opentelemetry-distro/src/amazon/opentelemetry/distro/_aws_metric_attribute_generator.py
Outdated
Show resolved
Hide resolved
| bucket_names: List[str] = [bucket["Name"] for bucket in s3_client.list_buckets()["Buckets"]] | ||
| put_bucket_name: str = "test-put-object-bucket-name" | ||
| if put_bucket_name not in bucket_names: | ||
| s3_client.create_bucket( | ||
| Bucket=put_bucket_name, CreateBucketConfiguration={"LocationConstraint": _AWS_REGION} | ||
| ) | ||
|
|
||
| get_bucket_name: str = "test-get-object-bucket-name" | ||
| if get_bucket_name not in bucket_names: | ||
| s3_client.create_bucket( | ||
| Bucket=get_bucket_name, CreateBucketConfiguration={"LocationConstraint": _AWS_REGION} | ||
| ) |
There was a problem hiding this comment.
Why is this needed? Same with changes to DDB, SQS sections? Is it because we are not refreshing the AWS container between tests?
There was a problem hiding this comment.
For each test, we will pull the "aws-application-signals-tests-botocore-app", and start a new container. Only the first container will succeed, all the following containers are fail with CreateBucket, exception:
[INFO] Unexpected exception occurred An error occurred (BucketAlreadyOwnedByYou) when calling the CreateBucket operation: Your previous request to create the named bucket succeeded and you already own it. Then the function directly return [None] without proceed with following code.
Same for other resources. So, for all the services, only the first container succeed with creating resources. All the following container fails. This was fine as long as we already created the resource before. But After adding kinesis_client consumer, we will have trouble, because we want to have different consumer_arn returned for each container.
There was a problem hiding this comment.
Interesting, thanks for investigating and improving!
Not blocking: Consider adding documentation to this logic that explains it really only runs once, to avoid confusion in future.
Add kinesis Consumer Support with the following change:
Modify patch
_KinesisExtensionwhere extractConsumerARNand add into"aws.kinesis.consumer_arn"span attributes. The attributes will be retrieved from both request and response context.Populate
RemoteResourceTypeandRemoteResourceIdentifierfor SNS in ADOT python and update unit test and contract test to verify ADOT python performance.Contract test -
botocore_server.py: As we only want to have one span per test, and the consumerArn contains random prefix, it will be different per test, we pre-computeconsumer_arnand pass it into RequestHandler. To avoid exception when creating resource, we check if the resource exist first, and create the resource when it not found. This make sureprepare_aws_serverfunction code can be run to the end without exception and theconsumer_arncan be returned.Contract test -
contract_test_base.py: AsConsumerARNcan include random prefix, we add_assert_match_attributefunction to check if the identifier match with certain pattern.Testing:
A manually E2E test is performed, and confirmed the expected traces metrics and service maps are generated:
Trace:
Metrics:
Service Map:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.