Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bedrock): add guardrails and evaluation jobs #162

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

npellegrin
Copy link
Contributor

Similar to #161 but requires upgrading AWS SDK version.

@npellegrin npellegrin force-pushed the feature/bedrock-v2 branch 2 times, most recently from 0ff33ca to 6194cbc Compare May 3, 2024 18:32
@npellegrin npellegrin marked this pull request as draft May 3, 2024 18:33
@ekristen
Copy link
Owner

@npellegrin are you going to have time to finish this up or should we close it for the time being?

@npellegrin
Copy link
Contributor Author

Sorry I did not realized this PR and #161 are pending. I have some time this week to look at it again, I will do some changes on it, test on a real account and flip the PR to "ready" once done 😃

@ekristen
Copy link
Owner

Awesome thank you so much.

@npellegrin
Copy link
Contributor Author

New resources:

  • BedrockGuardrail
  • BedrockEvaluationJob

Both support tags. Filtering is enabled for model evaluation already in "stopped" state.

Scan with feature activated:

us-east-1 - BedrockGuardrail - txq90g37kft9 - [ID: "txq90g37kft9", Name: "test", Status: "READY", Version: "DRAFT", tag:testtag1: "testvalue", tag:zzz: "zafafx"] - would remove
us-east-1 - BedrockEvaluationJob - test - [Arn: "arn:aws:bedrock:us-east-1:000000000000:evaluation-job/0mkxhxrmjqzi", Name: "test", Status: "Stopped", tag:testzz: "testaa"] - already stopped
Scan complete: 2 total, 1 nukeable, 1 filtered.

Delete with feature activated for guardrail:

us-east-1 - BedrockGuardrail - txq90g37kft9 - [ID: "txq90g37kft9", Name: "test", Status: "READY", Version: "DRAFT", tag:testtag1: "testvalue", tag:zzz: "zafafx"] - removed

Removal requested: 0 waiting, 0 failed, 1 skipped, 1 finished

Nuke complete: 0 failed, 1 skipped, 1 finished.

Delete with feature activated for model evaluation:

us-east-1 - BedrockEvaluationJob - test - [Arn: "arn:aws:bedrock:us-east-1:000000000000:evaluation-job/0mkxhxrmjqzi", Name: "test", Status: "InProgress"] - triggered remove

Removal requested: 1 waiting, 0 failed, 0 skipped, 0 finished

us-east-1 - BedrockEvaluationJob - test - [Arn: "arn:aws:bedrock:us-east-1:000000000000:evaluation-job/0mkxhxrmjqzi", Name: "test", Status: "InProgress"] - waiting

Removal requested: 1 waiting, 0 failed, 0 skipped, 0 finished

us-east-1 - BedrockEvaluationJob - test - [Arn: "arn:aws:bedrock:us-east-1:000000000000:evaluation-job/0mkxhxrmjqzi", Name: "test", Status: "InProgress"] - removed

@npellegrin npellegrin marked this pull request as ready for review November 18, 2024 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants